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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  Rotary-Wing  Aviation  Research  Unit  at  Fort 
Rucker,  Alabama,  is  committed  to  enhancing  the  readiness  of  Army 
aviation  units  through  development  of  effective  techniques  to 
train  operational  crews.  The  work  in  the  area  of  crew  coordina¬ 
tion  is  a  part  of  the  work  accomplished  for  meeting  this  objec¬ 
tive.  Considerable  effort  was  expended  to  examine  all  of  the 
available  information  on  methods  and  techniques  used  to  improve 
crew  performance.  The  research  described  in  this  report  is  the 
culmination  of  a  series  of  efforts  to  define  the  scope  of  the 
issues  and  empirically  verify  a  training  approach  to  improving 
crew  performance.  A  combined  effort  between  ARI  and  the  Direc¬ 
torate  of  Evaluation  and  Standardization  was  undertaken  to 
determine  the  effectiveness  of  crew  coordination  performance. 

Operational  AH-64  crews  flew  a  series  of  missions  under 
controlled  conditions.  Data  were  collected  regarding  the  suc¬ 
cessful  completion  of  those  missions  and  several  unexpected 
emergency  situations.  All  crews  flew  all  missions. 

The  results  showed  that  crews  trained  with  a  specially 
developed  set  of  training  materials  developed  by  ARI  outperformed 
crews  that  were  battle  rostered.  This  finding  indicated  that 
battle  rostering  does  not  solve  the  crew  coordination  problem. 

The  crew  coordination  training  did  result  in  improved  crew 
performance. 

These  findings  were  used  by  the  Army  aviation  community  in 
changing  the  long-standing  policy  that  required  AH-64  crews  to  be 
battle  rostered.  It  also  demonstrated  that  well-structured 
analyses  can  be  used  to  develop  the  training  requirements  neces¬ 
sary  to  satisfy  the  needs  of  aviation.  The  results  of  this  work 
were  briefed  to  the  Commanding  General,  U.S.  Army  Aviation  Cen¬ 
ter,  in  November  1993;  the  Commanding  General,  TRADOC,  in  January 
1994;  and  the  Chief  of  Staff,  U.S.  Army  (CSA),  in  March  1994. 

The  outcome  of  these  briefings  resulted  in  a  tasker  from  the  CSA 
to  Training  and  Doctrine  Command  to  examine  the  use  of  this  meth¬ 
odology  for  other  crewed  systems. 


EDGAR  M.  JOHNSON 
Director 
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EFFECT  OF  CREW  COMPOSITION  ON  AH-64  ATTACK  HELICOPTER  MISSION 
PERFORMANCE  AND  FLIGHT  SAFETY 


EXECUTIVE  SUMMARY 


Requirement : 

This  experiment  investigated  the  mission  performance  and 
flight  safety  effects  of  two  comparative  methods  of  improving 
crew  coordination:  standardized  training  and  battle  rostering. 


Procedure : 

U.S.  Army  AH-64  aviators  who  had  prior  crew  coordination 
training  and  recent  flight  experience  as  a  battle-rostered  crew 
were  paired  under  two  different  conditions.  Battle-rostered 
pairing  of  pilot  and  copilot  gunner  was  based  on  the  unit's  oper¬ 
ational  aircrew  designation.  Pilot  and  copilot  gunner  crew  mem¬ 
bers  for  nonbattle-rostered  (mixed)  crews  were  selected  randomly 
from  the  battle-rostered  crew  population.  Crews  completed  four 
different  evaluation  missions  of  comparable  stress  and  difficulty 
in  the  AH-64  combat  mission  simulator  (CMS) .  Subjects  flew  two 
evaluation  missions  as  a  battle-rostered  crew  and  two  evaluation 
missions  as  a  mixed  crew.  Battle-rostered  and  mixed  crew  mis¬ 
sions  were  counterbalanced  to  avoid  an  ordering  effect.  Each 
evaluation  mission  was  directly  observed  by  an  instructor  pilot 
(IP)  evaluator,  videotaped,  and  recorded  by  the  flight  simulator 
computer.  These  data  enabled  assessments  of  crew  behaviors,  task 
performance,  mission  performance,  flight  safety,  and  familiarity 
effect  for  each  crew  condition. 


Findings: 

Battle  rostering  is  often  perceived  to  improve  crew  perfor¬ 
mance.  However,  the  experimental  evidence  did  not  show  signifi¬ 
cant  improvements  in  mission  performance  or  flight  safety.  Crew 
performance  of  13  well-defined  crew  coordination  behaviors  (Basic 
Qualities)  and  25  aircrew  training  manual  (ATM)  tasks  produced 
mixed  results  with  no  statistically  significant  difference  be¬ 
tween  battle-rostered  and  mixed  crews.  Results  that  compared 
task  performance  with  crew  member  self-ratings  offered  sufficient 
evidence  to  suggest  that  battle-rostered  crews  tend  to  revert  to 
an  implicit  style  of  crew  coordination  more  often  than  mixed 
crews  do.  The  implicit  crew  coordination  tendency  demonstrated 
by  battle-rostered  crews  adversely  affected  their  performance  of 
some  ATM  tasks.  Although  there  were  small  but  statistically 
significant  differences  in  the  area  of  weapons  employment,  crew 
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performance  in  five  areas  of  mission  performance  (navigation, 
threat  avoidance,  evasion,  inflight  emergencies,  malfunctions) 
produced  mixed  results  with  no  other  statistically  significant 
difference.  Battle  rostering  crews  has  a  potential  positive 
effect  on  gunnery  performance. 

Crew  performance  of  flight  safety  intensive  mission  perfor¬ 
mance  areas  and  safety-related  ATM  tasks  produced  mixed  results 
with  no  statistically  significant  difference  between  battle- 
rostered  and  mixed  crews.  Since  mission  performance  was  similar 
in  either  crew  configuration,  the  finding  that  crew  members  rate 
themselves  50%  more  confident  when  flying  in  their  battle- 
rostered  crew  than  in  a  mixed  crew  has  negative  implications. 
Overconfidence  is  suspected  as  a  possible  contributor  to  aviation 
accidents  and  crew  complacency.  Battle-rostered  crews  rated 
their  workload  as  easier  and  their  crew  coordination  style  as 
more  implicit  than  mixed  crews. 


Utilization  of  Findings: 

The  results  of  the  experiment  lead  to  the  conclusion  that 
standardized  crew  coordination  training  alone  is  achieving  the 
Army's  objective  of  improving  crew  performance.  The  mechanism 
for  improving  crew  coordination,  the  Aircrew  Coordination  Ex¬ 
portable  Training  Package,  is  being  fielded  by  the  Army's  Avia¬ 
tion  Training  Brigade,  U.S.  Army  Aviation  Center,  Fort  Rucker, 
Alabama. 

Because  standardized  training  was  found  to  be  the  preferred 
solution  for  improving  crew  coordination,  it  is  now  more  impor¬ 
tant  than  ever  to  make  sure  the  training  is  fully  implemented. 
This  can  be  achieved  by  promulgating  in  key  policy  documents  the 
requirement  that  all  rated  and  nonrated  crew  members  must  com¬ 
plete  the  Army's  Aircrew  Coordination  Training  Program.  Tenden¬ 
cies  revealed  during  the  experiment  for  crew  members  to  revert  to 
an  implicit  crew  coordination  style  and  individual  rather  than 
crew  mission  orientation  make  it  important  to  develop  and  imple¬ 
ment  annual  crew  coordination  refresher  training.  To  fully 
implement  the  crew  coordination  training  and  evaluation  system, 
it  is  necessary  to  incorporate  the  crew  coordination  Basic 
Qualities  from  the  Aircrew  Coordination  Exportable  Training 
Package  into  each  ATM  for  annual  evaluations. 

Results  of  the  experiment  suggest  that  commanders  may  reduce 
risks  and  enhance  areas  of  mission  performance  by  battle  roster¬ 
ing  crews  for  short  periods  of  time  for  specific  missions,  for 
example,  gunnery.  Commanders  should  be  cautioned  that  making 
permanent  crew  assignments  can  potentially  jeopardize  both  mis¬ 
sion  performance  and  flight  safety. 
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EFFECT  OF  CREW  COMPOSITION  ON  AH-64  ATTACK  HELICOPTER  MISSION 

PERFORMANCE  AND  FLIGHT  SAFETY 


Introduction 

The  airline  industry  was  the  first  to  reveal  evidence 
implicating  crew  coordination  as  a  key  factor  in  a  number  of 
commercial  aviation  major  accidents  and  incidents.  It  was 
generally  accepted  that  improved  communication  and  coordination 
among  individual  crew  members  should  improve  flight  safety  and 
mission  effectiveness.  As  a  result,  alternative  means  were 
sought  to  achieve  better  coordination  among  crew  members.  This 
research  examines  two  comparative  means  of  improving  crew 
coordination:  battle  rostering  and  standardized  training. 

The  battle-rostering  approach  permanently  assigns  specific 
crew  members  together  as  a  crew.  For  example,  Kanki  and.Foushee 
(1989)  argued  that  commercial  airline  crews  performed  better, 
communicated  more  effectively,  and  made  fewer  errors  when  they 
had  recently  flown  together.  They  used  a  high-fidelity  flight 
simulator  to  observe  the  performance  of  airline  crews  that  had 
previously  flown  together  and  crews  that  had  not  flown  together. 
Their  analysis  focused  on  crew  interaction  as  indexed  by  the 
communication  process.  Their  research  concluded  that  recent 
operating  experience  generates  a  type  of  "familiarity, "  which  in 
turn,  fosters  improved  information  exchange  and  validation 
culminating  in  fewer  crew  errors. 

Povenmire,  Rockway,  Bunecke,  and  Patton  (1989)  found  that 
crew  coordination  performance  correlated  positively  with  B-52 
crew  tactical  maneuvers  and  bombing  accuracy.  B-52  crews  include 
six  permanently  assigned  crew  members.  Observations  were  made 
using  instruments  from  published  Cockpit  Resource  Management 
(CRM)  courses  and  mission  performance  outcome  measures  that  were 
specially  tailored  to  address  crew  task  performance.  Crews  were 
observed  conducting  operational  missions  in  the  B-52  Weapon 
System  Trainer.  The  research  concluded  that  crew  coordination 
training  enhances  combat  mission  performance  of  crews  that  fly 
together  on  a  permanent  basis. 

During  the  period  1988-1994,  the  U.S.  Army  Research 
Institute  (ARI)  conducted  a  program  of  training  research  that 
responds  to  the  Army's  need  for  better  crew  .coordination 
training.  This  program  of  research  was  conducted  in  close 
cooperation  with  the  U.S.  Army  Aviation  Center  (USAAVNC)  and  its 
efforts  to  revise  its  training  standards  to  reflect  increased 
emphasis  on  crew-level  performance. 

The  USAAVNC  formed  a  working  group  in  early  1990  to 
incorporate  the  results  of  the  aircrew  coordination  research  into 
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revisions  of  the  total  Aircrew  Training  Program  (Department  of 
the  Army,  1992b)  and  Aircrew  Training  Manuals  (ATMs)  for  all  Army 
aircraft.  The  March  1992  revised  Training  Circular  1-210 
introduced  battle  rostering  and  crew  coordination  as  policies 
designed  to  improve  effectiveness  and  safety  by  shifting  the 
training  emphasis  from  individual  to  crew-level  performance. 

Battle  rostering,  as  defined  in  TC  1-210  (Department  of  the 
Army,  1992b) ,  is  "the  designation  of  two  or  more  individuals  to 
perform  as  a  crew  and  is  fundamental  to  crew-level  training.  It 
(battle  rostering)  provides  for  the  development  and  proficiency 
of  critical  skills.  Crew  coordination  is  enhanced  by 
consistently  battle  rostering  the  same  crew  members."  (p.1-11) 

The  training  circular  also  states  the  purpose  and  definition  of 
crew  coordination  training.  "Aircrew  training  must  emphasize 
crew  coordination  as  a  vital  part  of  the  overall  training 
program.  It  (crew  coordination)  is  a  set  of  principles, 
attitudes,  procedures,  and  techniques  which  transforms 
individuals  into  an  effective  crew."  (p.1-4) 

The  new  crew  coordination  policy,  designed  to  standardize 
crew  behaviors,  required  development  of  a  training  course  of 
instruction.  Dynamics  Research  Corporation  (DRC)  worked  closely 
with  the  USAAVNC  working  group  to  draft  training  and  evaluation 
methods  and  materials  for  a  crew  coordination  validation  testbed 
effort.  During  the  1992  validation  testbed,  UH-60  crews  from 
Fort  Campbell,  KY  were  battle  rostered  by  their  units  for  the 
experiment.  Each  crew  completed  four  missions  in  the  visual 
flight  simulator  with  their  battle-rostered  crew  member.  The 
testbed  demonstrated  and  validated  the  new  field  exportable 
program  for  training  and  evaluating  crew  coordination  skills. 
Testbed  results  showed  that  the  crews  performed  their  missions 
significantly  more  effectively  and  safely  after  the  training  than 
before  the  training  (Simon  &  Grubb,  in  preparation) .  The  USAAVNC 
approved  the  Aircrew  Coordination  Exportable  Training  Package 
(ETP)  (Pawlik,  Simon,  Grubb,  &  Zeller,  1992)  late  in  1992  and 
began  implementing  it  Army-wide  the  following  year. 

Since  promulgation  of  TC  1-210  (Department  of  the  Army, 

1992b) ,  anecdotal  data  and  aviation  accident  investigation 
reports  suggested  that  battle  rostering  compromised  operational 
safety.  Discussions  with  U.S.  Army  Safety  Center  accident 
investigators  indicated  complacency  and  overconfidence  were 
possible  contributing  factors.  The  U.S.  Air  Force  conducted 
research  comparing  the  effects  of  fixed  versus  formed  aircrews  on 
military  transport  accident  rates.  The  research  showed  that 
mixed  crews  (as  opposed  to  battle-rostered  crews)  were 
significantly  safer  (Woody,  McKinney,  Barker,  &  Clothier,  1994) . 
The  U.S.  Navy  Safety  Center  conducted  a  six-month  analysis  of 
naval  aviation  accident  data  to  find  out  the  effects  of  crew 
composition.  Discussions  with  U.S.  Navy  Safety  Center  analysts 
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revealed  that  the  results  did  not  provide  sufficient  evidence  to 
recommend  a  crewing  policy.  The  USAAVNC  notified  all  Army 
aviation  units  to  defer  crew  readiness  level  progression  and  crew 
training  pending  explicit  research  results. 

By  the  spring  of  1993  it  was  becoming  clear  that  two  methods 
of  improving  crew  coordination  were  available  to  the  Army. 

First,  the  crew  coordination  improved  through  standardized 
instruction  of  crew  members  in  a  set  of  coordination  behaviors 
and  skills.  In  this  case  standardization  of  behavior  was  an 
essential  ingredient.  Second,  battle  rostering  was  believed  to 
improve  crew  coordination  by  allowing  crew  members  to  develop 
familiar  patterns  of  interaction  and  common  understanding.  With 
battle  rostering  it  was  possible  that  more  idiosyncratic  behavior 
would  develop. 

While  these  two  methods  were  available,  little  was  known 
regarding  their  relative  effectiveness.  Constraints  on  the 
validation  experiment  conducted  earlier  with  the  new  field  ETP 
(Pawlik  et  al.,  1992)  did  not  permit  a  comparative  examination  of 
battle  rostering  versus  crew  coordination  training.  On  the  other 
hand,  anecdotal  evidence  suggested  that  some  crews  were 
benefiting  from  the  Army's  battle-rostering  policy;  yet,  these 
same  crews  had  not  yet  received  the  crew  coordination  training. 
Clearly,  a  comparative  evaluation  of  battle  rostering  versus  crew 
coordination  training  was  needed  to  clarify  the  Army's  policies 
in  this  area. 

Accordingly,  the  USAAVNC  asked  ARI  in  the  spring  of  1993  to 
conduct  such  a  comparative  evaluation.  Since  crew  coordination 
training  had  already  been  shown  to  produce  significant 
improvements  (Simon  &  Grubb,  in  preparation),  and  since  the  Army 
had  already  begun  to  field  this  training,  the  USAAVNC  decided 
that  this  evaluation  should  consider  battle  rostering  as  a  second 
variable.  Thus,  an  experiment  was  designed  to  examine  the 
contributions  of  battle  rostering  to  crew  performance.  In  this 
context,  the  present  study  addresses  five  hypotheses  organized 
along  different  dimensions  of  crew  performance: 

1.  Crew  coordination  behavior  will  be  significantly 
improved  through  a  combination  of  crew  coordination  training  and 
battle  rostering,  as  compared  to  the  use  of  crew  coordination 
training  alone. 

2.  Crew  flight  task  performance  will  be  significantly 
improved  through  a  combination  of  crew  coordination  training  and 
battle  rostering,  as  compared  to  the  use  of  crew  coordination 
training  alone. 
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3.  Mission  performance  will  be  significantly  improved 
through  a  combination  of  crew  coordination  training  and  battle 
rostering,  as  compared  to  the  use  of  crew  coordination  training 
alone . 

4.  Flight  safety  will  be  improved  through  a  combination  of 
crew  coordination  training  and  battle  rostering,  as  compared  to 
the  use  of  crew  coordination  training  alone. 

A  fifth  hypothesis  addresses  those  concerns  specifically 
raised  by  recent  accident  experience  with  battle-rostered  crews: 

5.  Crew  overconfidence  and  complacency  will  be  observed 
more  frequently  with  battle-rostered  crews,  as  compared  to 
nonbattle-rostered  crews. 

This  report  describes  the  research  conducted  to  compare  the 
impact  of  battle  rostering  and  crew  coordination  training  on  crew 
performance.  The  data  was  collected  on  operational  crews  under 
carefully  controlled  conditions  performing  specific  ATM  tasks  in 
a  simulated  tactical  situation. 

Method 

The  research  method  included  the  use  of  a  repeated  measures 
design  to  compare  crew  performance  effects  under  two  different 
crew  composition  conditions:  battle-rostered  and  mixed  crews. 

The  repeated  measures  design  was  counterbalanced  across  four 
mission  scenarios  and  the  two  crew  conditions.  In  addition  to 
accounting  for  differences  in  mission  scenarios  and  possible 
order  and  practice  effects,  the  design  provided  two  observations 
and  multiple  data  points  for  each  crew  condition.  Essential 
details  of  all  aspects  of  the  research  method  are  described  in 
this  section  of  the  report. 

Design 


A  repeated  measures  experimental  design,  counterbalanced 
across  four  tactical  mission  scenarios  and  two  crew 
configurations  was  developed  for  the  research.  Table  1 
graphically  portrays  the  experimental  design. 
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Table  1 


Experimental  Design 


Crews 

Scenarios 

1 

2 

3 

4 

1-4 

9  &  10 

Battle 

rostered 

Battle 

rostered 

Mixed 

Mixed 

5-8 

11  &  12 

Mixed 

Mixed 

Battle 

rostered 

Battle 

rostered 

Repeated  measures  was  especially  suitable  to  account  for  any 
minor  differences  in  difficulty  among  the  four  scenarios  while 
providing  multiple  data  points  from  a  reduced  number  of 
participating  aircrews.  The  design  provided  two  observations  for 
the  battle-rostered  crew  configuration  and  two  observations  for 
the  mixed  crew  configuration  to  address  the  strength  of  effect. 
Counterbalancing  these  two  crew  configurations  served  to  account 
for  possible  order  and  practice  effects  among  participant 
aircrews  and  individual  crew  members. 

Operational  constraints  required  minor  modification  of  a 
preferred  experimental  design  to  accommodate  aircrew  and 
simulator  availability  and  unit  training  requirements.  Principal 
among  the  constraints  was  that  crews  were  scheduled  for  data 
collection  missions  by  company  rather  than  randomly  across  the 
battalion.  This  modification  was  made  to  permit  company 
integrity  for  unit  training  during  the  three  weeks  of  the 
experiment.  Consequently,  it  was  necessary  to  substitute  two 
copilot  gunners  in  mixed  crews  for  four  missions  to  implement  the 
counterbalanced  design.  The  substitution  of  technically  current 
and  crew  coordination  trained  copilot  gunners  in  mixed  crews  only 
had  no  effect  on  the  results  obtained  from  the  repeated  measures 
design.  In  all,  7  of  the  24  mixed  crews  were  formed  from  across 
companies  rather  than  from  within  companies.  Forming  mixed  crews 
from  within  companies  was  consistent  with  the  unit's  standard 
procedures  but  introduced  into  the  experiment  an  aspect  of 
potential  difference  in  crew  member  familiarity.  The  AH-64 
combat  mission  simulator  (CMS)  was  available  only  for  13  days  of 
data  collection,  eliminating  the  possibility  of  more  than  four 
observations  per  crew. 

Each  of  the  12  crews  completed  each  scenario  to  produce  48 
data  collection  missions.  Participating  aviators  conducted  two 
missions  in  a  battle-rostered  crew  and  two  missions  in  a  mixed 
crew.  Crew  exposure  to  the  evaluation  scenarios  was 
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counterbalanced;  that  is,  crews  given  evaluation  scenarios  one 
and  two  for  the  battle-rostered  missions,  were  given  scenarios 
three  and  four  for  the  mixed  missions,  and  vice  versa. 

Subjects 


Thirty-three  aviators  rated  in  the  AH-64  Apache  attack 
helicopter  were  selected  to  participate  in  the  experiment. 
Participants  included  standardization  instructor  pilots  (SIP) , 

CMS  operators,  and  subject  aircrews. 

Two  SIPs  from  the  USAAVNC's  Directorate  of  Evaluation  and 
Standardization  (DES)  and  one  Crew  Coordination  Cadre  Training 
Team  SIP  were  combined  with  unit  instructor  pilots  (IP)  to 
evaluate  crew  performance.  All  five  of  these  IP  evaluators  were 
selected  based  on  their  training  and  experience  in  using  the 
Army's  crew  coordination  evaluation  procedures.  Two  AH64CMS 
subject  matter  experts  were  selected  to  program  and  control  the 
evaluation  mission  scenarios. 

Twelve  battle-rostered  aircrews,  each  consisting  of  a  pilot 
and  copilot  gunner  (CPG) ,  were  identified  by  their  unit  as 
subject  aircrews  for  the  experiment.  The  experiment  specified 
that  subject  aircrews  meet  two  requirements:  (a)  each  crew  member 
had  completed  training  in  the  Army's  Aircrew  Coordination 
Exportable  Training  Package  (Pawlik  et  al.,  1992),  and  (b)  the 
subject  aircrew  had  flown  at  least  one  mission  as  a  battle- 
rostered  crew  in  the  60  days  prior  to  the  experiment. 

The  initial  set  of  12  aircrews  consisted  of  four  battle- 
rostered  crews  from  each  of  the  battalion's  three  attack 
helicopter  companies.  Battle-rostered  crew  pairings  of  pilot  and 
CPG  were  based  on  the  unit's  operational  aircrew  designation. 

Crew  members  for  mixed  crews  were  selected  randomly  within  each 
company. 

Unit  operational  and  training  requirements  made  it  necessary 
to  substitute  two  CPG  for  the  primary  crew  member  in  four  of  the 
mixed  crews.  These  alternate  subjects  fully  met  the  requirements 
of  the  experiment.  The  final  subject  aircrew  pool  consisted  of 
12  pilots  and  14  CPG.  Three  subject  aircrew  members  were  not 
available  for  the  exit  interview  and  self-evaluation  phase  of  the 
experiment  due  to  temporary  duty  assignments.  Selected 
demographic  information  for  participants  in  .the  experiment  is 
shown  in  Table  2 . 
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Table  2 


Experiment  Participants 


Number 

Ranks 

AH-64  flight  hours 

IP  evaluators 

5 

MAJ 

(1)  , 

CW4 

(1), 

Range:  500-2000 

CW3 

(2)  , 

CW2 

(1) 

Average:  1180 

CMS  operators 

2 

CW3 

(1)  , 

CIV 

(1) 

Range:  1000-1540 
Average:  1270 

Crew  members 

-  Pilot 

12 

CW3 

(1)  , 

CW2 

(ID 

Range:  145-1200 
Average:  500 

-  CPG 

14a 

CPT 

(3)  , 

LT 

(3)  , 

Range:  70-1000 

CW4 

(2)  , 

CW3 

(2)  , 

Average:  300 

CW2 

(4) 

includes  2  substitute  CPG  in  4  mixed  crew  missions. 


Equipment 

AH 6 4 CMS .  The  AH64CMS  is  a  simulation  system  designed  for 

training  in  the  use  of  AH-64  Apache  helicopters.  The  CMS 
simulates  the  Apache  helicopter  and  its  related  systems  to  the 
same  level  of  performance  as  found  in  the  operational  systems. 

The  CMS  consists  of  two  separate  compartments  for  the  pilot  and 
copilot  gunner,  each  having  a  six-degree-of-freedom  hydraulic 
motion  system.  Each  compartment  includes  a  crew  member  station, 
pilot  or  copilot  gunner,  in  the  forward  portion  and  instructor/ 
operator  and  observer  stations  in  the  rear. portion.  Each 
compartment  is  equipped  with  a  visual  system  that  simulates 
natural  helicopter  environment  surroundings.  The  CMS  provides 
normal  and  emergency  procedural  mission  and  weapons  delivery 
training.  Additional  capabilities  include  navigation,  instrument 
flight  operation,  day,  dusk,  and  night  visual  flight  operations, 
ordnance  delivery,  and  aircraft  survivability  systems  of  the 
attack  helicopter.  In  addition,  the  CMS  can  be  used  to  simulate 
tactical  threat  systems. 

Video  recording  and  playback.  Each  simulator  mission  was 
observed  and  recorded  using  four  video  cameras  multiplexed  onto 
one  video  picture  (Figure  1)  .  All  intercom,  radio 
communications,  and  aural  warnings  were  recorded  onto  the 
videotape.  One  camera  was  placed  in  each  simulator  compartment 
and  aimed  to  provide  a  high-over-the-shoulder  view  of  each  crew 
member.  Two  cameras  were  placed  in  the  instructor/operator 
station  of  the  copilot  gunner  compartment  and  aimed  at  monitors 
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to  capture  each  crew  member's  visual  field  of  view  and  symbology 
overlay.  Videotape  players  and  monitors  were  located  in  crew 
after  action  review,  IP  evaluator,  and  data  collector  areas  to 
review  crew  performance  after  each  mission.  A  notebook  computer 
with  data  logger  software  designed  for  the  experiment  was  used  to 
synchronize  data  collector's  written  comments  with  the  mission 
time  overprinted  on  the  videotape. 

Materials 


Training .  The  training  and  evaluation  components  of  the 
Aircrew  Coordination  Exportable  Training  Package  (Pawlik,  et  al., 
1992)  were  implemented  as  published.  These  training  materials 
are  described  in  two  separate  reports  previously  developed  for 
ARI  (Grubb,  Simon,  Leedom,  &  Zeller,  1993;  Pawlik,  Simon,  Grubb, 

&  Zeller,  in  preparation) . 

Scenarios .  Four  attack  helicopter  tactical  scenarios  were 
developed  for  the  crew-level  evaluations  to  assess  changes  in 
crew  mission  performance  and  flight  safety.  Scenario  objectives 
and  tasks  were  designed  to  present  four  equally  difficult 
missions  to  the  aircrews.  The  data  collection  scenarios  included 
various  combinations  of  the  mission  segments  and  activities  shown 
in  Table  3. 

Each  of  the  four  evaluation  scenarios  was  designed  to  test 
the  subject  aircrews  on  their  ability  to  plan  and  execute  a 
typical  attack  helicopter  mission  at  night  in  a  mid-intensity 
threat  environment.  Each  scenario  involved  multiple  AH-64 
aircraft  with  the  evaluated  crew  acting  as  either  company  or 
heavy  team  lead.  The  scenarios,  lasting  approximately  an  hour 
and  fifteen  minutes  each,  included  numerous  crew  coordination 
related  ATM  tasks  and  required  the  crew  members  to  interact  with 
one  another  to  successfully  accomplish  the  mission. 

Scenarios  one  and  three  originated  in  forward  assembly  areas 
and  required  the  crew  to  navigate  to  a  forward  arming  and 
refueling  point  (FARP)  for  ammunition  and  fuel  before  proceeding 
to  prebriefed  battle  positions.  En  route  to  the  FARP,  the  crews 
experienced  a  minor  malfunction  of  an  aircraft  system,  which  was 
administratively  corrected  once  they  arrived  at  the  FARP.  After 
occupying  two  different  battle  positions  and  engaging  a  variety 
of  armor,  artillery,  air  defense,  and  mechanized  targets,  the 
crews  encountered  deteriorating  weather  conditions  while 
returning  to  the  FARP  and  inadvertently  entered  instrument 
meteorological  conditions.  The  missions  ended  when  the  crew 
completed  a  non-precision  instrument  approach  to  a  recovery 
airfield. 
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Data  Collection  Scenario  Outlines 
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refuel/ 


The  other  two  scenarios  started  in  holding  areas  with  the 
aircraft  fully  loaded  with  fuel  and  ammunition.  In  scenario  two, 
the  crews  navigated  to  a  forward  assembly  area  to  receive  a 
mission  update  before  proceeding  to  prebriefed  battle  positions. 
In  the  initial  battle  position,  the  crews  engaged  armor, 
mechanized,  and  air  defense  targets  before  proceeding  to  a 
successive  battle  position.  While  navigating  between  battle 
positions,  the  crews  experienced  a  minor  malfunction  of  an 
aircraft  system.  Returning  to  the  FARP  at  the  end  of  the 
mission,  the  crews  experienced  a  major  malfunction  of  an  aircraft 
system. 

In  scenario  four,  the  crews  navigated  directly  to  their 
initial  battle  position.  There,  the  crews  engaged  armor, 
mechanized,  and  air  defense  targets  before  being  ordered  to 
accompany  the  scout  aircraft  to  confirm  a  suspected  enemy 
airmobile  assault  behind  friendly  lines.  While  en  route  to  the 
second  battle  position,  the  crews  experienced  a  minor  malfunction 
of  an  aircraft  system.  In  the  second  battle  position,  the  crews 
engaged  light  armored  vehicles  and  attack  and  utility  helicopters 
conducting  an  airmobile  assault.  While  returning  to  the  FARP  at 
the  end  of  the  mission,  the  crews  experienced  a  major  malfunction 
of  an  aircraft  system. 

Rating  instruments  and  interview  guide.  Measures  similar  to 
those  previously  used  for  the  utility  helicopter  validation 
testbed  (Simon  &  Grubb,  in  preparation)  were  developed  and  tested 
to  confirm  their  accuracy  in  data  collection  using  the  AH64CMS. 
Crew  behaviors  (Basic  Qualities),  ATM  task  performance,  mission 
performance,  and  familiarity  effects  measures  are  defined  later 
as  the  dependent  variables  to  address  experiment  hypotheses. 
Following  is  a  description  of  the  rating  instruments  and 
interview  guide  for  measures  used  in  the  experiment. 

Two  forms  from  the  Aircrew  Coordination  Exportable  Training 
Package  (Pawlik  et  al.,  1992)  were  used  to  record  crew  behaviors 
and  performance:  (a)  DA  Form  7121-R  (Department  of  the  Army, 
1992a)  entitled  Battle-Rostered  Crew  Evaluation/Training  Grade 
Slip  and  (b)  the  Aircrew  Coordination  Training  Grade  Slip  based 
on  DA  Form  5882-R  (Department  of  the  Army,  1992a)  entitled 
Maneuver/Procedure  Grade  Slip  for  AH-64  Aviators. 

The  Battle-Rostered  Crew  Evaluation/Training  Grade  Slip 
(Pawlik  et  al.,  1992)  allowed  IP  evaluators  to  make  written 
comments  and  record  a  grade  for  the  overall  flight  (Figure  2) . 
More  detailed  information  was  recorded  and  attached  by  way  of  the 
Aircrew  Coordination  Training  Grade  Slip  (Pawlik  et  al.,  1992), 
(Figure  3) ,  which  provided  IP  evaluators  with  space  for  crew 
information,  multiple  entries  for  grading  each  task,  and  a  look¬ 
up  table  of  crew  coordination  behaviors  at  the  bottom  of  each 
page.  A  grade  block  was  provided  to  enter  a  summary  rating  for 
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BATTLE-ROSTERED  CREW  EVALUATION/TRAINING  GRADE  SUP 

For  use  of  this  form,  see  Aircraft  ATM;  the  proponent  agency  is  TRADOC 

BATTLE-  NAME  RANK 

ROSTERED  PC: _ _ _ _ _ 

CREW 

EXAMINEES/  _ - — - - - 

TRAINEES  NONRATED  CREW  MEMBERS 

DUTY  SYMBOL  NAME  RANK 


EVALUATOR/ 

INSTRUCTOR 


CREW  DATA 


TOTAL  BATTLE-ROSTERED 
CREW  HOURS 

PURPOSE:  EVALUATION/TRAINING 

TIME  TODAY: _ 

TYPE  AIRCRAFT: 


CREW  TASK  1 

D/N/NVD 

CREW  TASK  2 

D/N/NVD 

CREW  TASK  3 

D/N/NVD 

CREW  TASK  4 

D/N/NVD 

CREW  TASK  5 

D/N/NVD 

NIGHT 

WX 

DATE  DESIGNATED  A  BATTLE- 
ROSTERED  CREW: 


I  CUMULATIVE  TIME: _ 

CREW  TASK  6  _ D/N/NVD 

CREW  TASK  7  _ D/N/NVD 

CREW  TASK  8  _ D/N/NVD 

CREW  TASK  9  _ D/N/NVD 

CREW  TASK  10 _ D/N/NVD 

SIMULATOR _ NVG _ 


_ EVALUATOR/INSTRUCTOR  RECOMMENDATIONS _ 

1=1  (ISSUE)  (VALIDATE)  CREW  QUALIFICATIONS 

1=1  (SUSPEND)  (REVOKE)  CREW  QUALIFICATIONS 

O  REQUIRES  ADDITIONAL  (FLIGHT)  (ACADEMIC)  (SIMULATION  DEVICE)  TRAINING 
^  SEE  BACK  FOR  COMMENTS 

I  HAVE  DEBRIEFED  THE  EXAMINEES/TRAINEES  AND  INFORMED  THEM  OF  THEIR  STATUS. 

EVALUATOR'S/INSTRUCTOR’S  SIGNATURE: _ _ 

WE  HAVE  BEEN  DEBRIEFED  BY  THE  EVALUATOR/INSTRUCTOR  AND  UNDERSTAND  OUR 
CURRENT  STATUS. 

PCS  SIGNATURE: _ _ _ _ 

PI’S  SIGNATURE: _ _ _ 

NONRATED  CREW  MEMBER’S  SIGNATURES:  _ _ 


OVERALL  GRADE  FOR  THIS  FLIGHT  IS:  S  U  NA 
DA  FORM  7121-R,  MAR  92 


Figure  2.  Battle-rostered  crew  evaluation/training  grade  slip 
(page  1  of  2) . 
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COMMENTS 


PAGE  2,  DA  FORM  7121-R.  MAR  92 


Figure  2.  Battle-rostered  crew  evaluation/training  grade  slip 
(page  2  of  2)  . 
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NO. 

STAN  EVAL/TRAINING  TASKS 

37 

IHADSS  OPERATIONS 

38 

DATA  ENTRY  PROCEDURES  (FS) 

39 

ACFT  POSrTION  UPDATE  (FS) 

40 

TARGET  STORE  £FS) 

41 

FIRING  POSITION  OPS 

42 

ENGAGE  TGT  WITH  HELLFIRE 

43 

ENGAGE  TGT  WITH  ARCS 

44 

ENGAGE  TGT  WITH  AWS 

46 

WPNS  INITIALIZATION 

46 

TARGET  HANDOVER 

47 

IHADSS  TARGET  TRACKING 

48 

ORAL  EVALUATION 

MANEUVER/PROCEDURE  GRADE  SUP  FOR  AH-64  AVIATORS 

For  use  of  this  form,  see  Aircrew  Coordination  Exportable  Training  Package  and  TC  1-214 

P _  Date  _ 

CPG _ 

Instructor  or  evaluator  will  sign  in  the  first  unused  block  of  each  area  trained  or  evaluated 


STAN  EVAL/TRAINING  TASKS 


CREW  MISSION  BRIEFING 


PLAN  VFR  FLIGHT 


DA  FORM  6701 -fl  (PPC) 


PREFLIGHT  INSPECTION 


ENG  START,  RUN-UP,  T/0  CKS 


HOVER  POWER  CHECK 


NORMAL  TAKEOFF 


TRAFFIC  PATTERN  FLIGHT 


FUEL  MANAGEMENT  PROCEDURES 


DOPPLER  NAVIGATION  (FS) 


BEFORE  LANDING  CHECK 


VMC  APPROACH 


CONFINED  AREA  OPERATIONS 


SLOPE  OPERATIONS 


TERRAIN  FLIGHT  TAKEOFF 


TERRAIN  FLIGHT 


NOE  DECELERATION 


TERRAIN  FLIGHT  APPROACH 


STANDARD  AUTOROTATION 


SIM  SINGLE  ENG  FAILURE  ALT 


SIM  SINGLE  ENG  FAILURE  OGE 


SINGLE-ENGINE  LANDING 


ECU  LOCKOUT  OPERATIONS 


TERRAIN  FLIGHT  NAVIGATION 


EMERGENCY  PROCEDURES 


IIMC  PROCEDURES/VHIRP 


MASKING  AND  UNMASKING  (BS) 


MAJ  US/ALLi  ED /THREAT  EQUIP 


ACFT  SUHVIV  EQUIP  (BS) 


AFTER-LANDING  TASKS 


MARK  XD  IFF  SYSTEM 


TADS  OPERATIONAL  CHKS  (FSI 


TADS  BO  RE  SIGHTING  (FSI 


TADS  SENSOR  OPERATIONS  (FS) 


IHADSS  BO  RE  SIGHT 


IHADSS  VIDEO  ADJUSTMENTS 


Ml GHT/NVD /EVAL/TRAINING  TASKS 


CREW  MISSION  BRIEFING 


NVS  OPERATIONAL  CHECKS 


GROUND  TAXI 


HOVER  POWER  CHECK 


HOVERING  FLIGHT 


NORMAL  TAKEOFF 


ROLLING  TAKEOFF  (BSI 


TRAFFIC  PATTERN  FLIGHT 


FUEL  MANAGEMENT  PROCEDURES 


PILOTAGE  &  DEAD  RECKONING 


DOPPLER  NAVIGATION 


VMC  APPROACH 


CONFINED  AREA  OPERATIONS 


SLOPE  OPERATIONS 


TERRAIN  FLIGHT  TAKEOFF 


TERRAIN  FLIGHT 


NOE  DECELERATION 


TERRAIN  FLIGHT  APPROACH 


SIM  SINGLE-ENG  FAILURE  ALT 


SINGLE-ENGINE  LANDING 


AIRCREW  COORDINATION  BASIC  QUALITIES 


WORK 
10A0  I  EV0fT5 


12.  13. 

ADVOO  AAR 

assott 


AIRCREW  COORDINATION  TRAINING  GRADE  SUP 


Figure  3.  Aircrew  coordination  training  grade  slip 
(page  1  of  2)  . 
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MANEUVER/PROCEDURE  GRADE  SUP  FOR  AH-64  AVIATORS 


NIGfflTNVD  EVAUTRAJNING  TASKS 


TERRAIN  FUGKT  NAVIGATION 


EMERGENCY  PROCEDURES 


UNUSUAL  ATTITUDE  RECOVERY 


IIMC  PROCEDURES A/H I RP 


MASKING  &  UNMASKING  (BSI 


TADS  OPERATIONAL  CKS  (PS) 


TADS  BO  RESIGHT  (FS) 


TADS  SENSOR  OPERATIONS  (FS) 


IHADSS  BORESIGHTING 


IHADSS  VIDEO  ADJUSTMENTS 


IHADSS  OPERATIONS 


NVG  FAILURE  (FS) 


ORAL  EVALUATION 


INST  EVAL/TRAINING  TASKS 


CREW  MISSION  BRIEFING 


IFR  FUGHT  PLANNING 


ENG  START.  RUNUP.  T/O  CKS 


FUEL  MANAGEMENT 


BEFORE-LANDING  CHECK 


SIM  SINGLE-ENG  FAILURE  ALT 


EMERGENCY  PROCEDURES 


RADIO  NAVIGATION 


HOLDING  PROCEDURES 


UNUSUAL  ATTITUDE  RECOVERY 


RADIO  COMMUNICATIONS  PROC 


NONPRECISION  APPROACH 


PRECISION  APPROACH 


IIMC  PROCEDURES/VH1RP 


AFTER-LANDING  TASKS 


ORAL  EVALUATION 


TRNG/MISSlON/ADDmONAL  TASKS 


DO  FORM  366-4 


SIM  MAX  PERF  T/O  (BS) 


DECELERATION/ACCEL  (BS) 


ROLL-ON  LANDING  (BS) 


TERRAIN  FLT  MISSION  PLAN 


HIGH-SPEED  FLIGHT 


SIM  SINGLE  ENG  FAILURE  IGE 


STABILATOR  MALF  PROCEDURE 


INSTRUMENT  TAKEOFF  (BS) 


TWO-WAY  RADIO  FAILURE 


PINNACLE  OR  RIOGEUNE  OPS 


HIGH/LOW  G  FUGHT 


EVASIVE  MANEUVERS  (BS) 


MULTIAIRCRAFT  OPERATIONS 


CALL  FOR/ADJUST  INO  FIRE 


VIS  SIGNAL  TECHNIQUES  (FS) 


LASER  SPOT  TRACKER  OPS  (FS) 


FARP  PROCEDURES 


ACTIONS  ON  CONTACT 


ID  TARGETS  WITH  TADS  (FSI 


SELECT  APPROPRIATE  WPN  SYS 


TGT  TRACKING  WITH  TADS  (FS) 


OPERATE  ONBD  RECORDER  (FS) 


DIVING  FUGHT  (BS) 


TECHNIQUES  OF  MOVEMENT  (BSI 


NEGOTIATE  WIRE  OBSTACLES 


TACTICAL  COMMO  AND  ECCM 


TRANS  TACTICAL  REPORT  (FS) 


NOTES: 

•  REQUIRED  FOR  NVG  EVAL 
+  REQUIRED  FOR  NIGHT  EVAL 

ENTER  S+.  S,  S*.  ORU  IN  GRADE  BLOCK.  IF  GRADE  IS  S- 
OR  U  DUE  TO  AIRCREW  COORDINATION  INCLUDE  BASIC 
QUALITY  NUMBER(S) 


I.  1 

CREW  PLAN 

CU-  RE- 

MATE  >«ARS€ 


AIRCREW  COORDINATION  BAjSXT  QUALITIES 


s. 

«. 

7. 

S. 

i. 

ia 

11. 

12. 

UNEXP 

9TO 

SJT 

COMM 

CROSS 

INFO 

ADVOO 

EVENT* 

XFE* 

AWARE 

ACX 

SOUGHT 

MON¬ 

OF¬ 

assstt 

ITOR 

FERED 

PAGE  2,  AIRCREW  COORDINATION  TRAINING  GRADE  SUP 


Figure  3.  Aircrew  coordination  training  grade  slip 
(page  2  of  2) . 
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each  crew  coordination  behavior  at  the  bottom  of  the  last  page. 
For  purposes  of  this  report,  these  two  forms  are  collectively 
referred  to  as  the  grade  slip. 

Based  on  previous  research,  the  USAAVNC  defines,  trains,  and 
evaluates  crew  coordination  behaviors  in  terms  of  13  Basic 
Qualities.  Each  Basic  Quality  was  rated  by  an  IP  evaluator  on  a 
seven-point  scale  and  recorded  on  the  grade  slip.  The  seven- 
point  scale  was  anchored  at  the  1,  4,  and  7  levels  with  specific 
behavioral  descriptions  of  performance  at  those  levels.  IP 
evaluators  were  instructed  to-  interpolate  ratings  of  2,  3,  5,  and 
6  from  the  descriptions  given  at  the  1,  4,  and  7  levels  as  being 
somewhat  better  or  worse  than  the  anchored  description.  The 
numbers  associated  with  the  Basic  Quality  ratings  were  1  =  Very 
Poor,  2  =  Poor,  3  =  Marginal,  4  =  Acceptable,  5  =  Good,  6  =  Very 
Good,  and  7  =  Superior.  The  Basic  Quality  definitions,  anchor 
descriptions,  and  detailed  rating  guidelines  are  provided  in  the 
Aircrew  Coordination  Exportable  Training  Package  (Pawlik  et  al., 
1992)  . 

Aircrew  tasks  contained  in  the  ATM  for  the  AH-64,  TC  1-214 
(Department  of  the  Army,  1992a)  were  used  as  basis  for 
determining  crew  performance.  All  ATM  tasks  include  both  a  crew 
coordination  and  a  technical  flight  skill  component.  IP 
evaluators  rated  crews  on  various  ATM  tasks  but  were  required  to 
rate  crews  on  25  selected  tasks  for  each  evaluation  mission.  ATM 
task  performance  was  graded  on  a  four-point  scale:  S+,  S,  S-, 
and  U.  When  a  crew  received  a  grade  of  S-  or  U  due  to  crew 
coordination,  IP  evaluators  noted  on  the  grade  slip  which  Basic 
Qualities  contributed  negatively  to  the  task  grade. 

Scenario  worksheets  were  developed  as  a  means  for  IP 
evaluators  to  record  crew  mission  performance  ratings.  The 
worksheets  were  designed  to  follow  the  segments  of  each  scenario 
(Figure  4) .  Circle  or  fill-in  type  entries  were  used  to 
facilitate  rapid  and  accurate  recording  of  IP  evaluator  entries 
in  the  following  categories  of  crew  mission  performance: 
navigation,  weapons  employment,  threat,  unexpected  events,  and 
instrument  approach.  Rating  scales  describing  levels  of 
performance  within  each  mission  category  were  used  to  measure 
crew  performance.  For  example,  the  rating  scale  for  the  planning 
dimension  of  the  instrument  approach  category  describes  four 
levels  of  performance.  A  superior  rating  required  both  crew 
members  to  review,  discuss,  and  rehearse  the  approach.  A  good 
rating  was  achieved  when  one  crew  member  reviewed  the  approach, 
briefed  the  other  crew  member,  and  then  talked  him  through  the 
approach.  A  poor  rating  was  assigned  a  crew  where 
one  crew  member  reviewed  the  approach  and  talked  the  other 
through  it.  Finally,  an  unsatisfactory  rating  was  assigned  when 
one  crew  member  reviewed  and  executed  the  approach  without 
assistance. 
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AH-64  SCENARIO  #1  (CMS  PHASE  II) 


SEGMENT  5 :  Movement  from  the  BP  to  the  FARP 

DESCRIPTION:  The  segment  begins  as  the  crew  departs  the  successive  BP  enroute 
to  the  FARP.  During  this  segment,  the  crew  encounters  an  inadvertant  entry  into 
instrument  meteorological  conditions  (IMC) .  The  crew  must  then  plan  and  execute 
a  nonprecision  instrument  approach  to  transition  back  to  visual  meteorological 
conditions  (VMC) .  The  segment  ends  when  the  crew  completes  a  safe  landing. 


TASK  1083  Perform  inadvertent  IMC  procedures/VHIRP 

GRADE:  S  +  S  S-  U 

Basic  Qualities:  _ ,  _ 

NOTES: 


-  VHIRP  steps  accomplished?  _  of  5 

-  Successful  transition  from  VMC  to  IMC?  Yes/No  (Circle  one) _ _ _ 

Crew  approach  planning  (Circle  one) 

3  Both  crew  members  review,  discuss,  and  rehearse  the  approach 

2  One  crew  member  reviews  the  approach,  briefs  the  other  prior  to  executing 
the  approach,  then  talks  the  other  through  it 

1  One  crew  member  reviews  the  approach  and  talks  the  other  through  it 

0  One  crew  member  reviews  and  executes  the  approach  with  no  assistance  from 

the  other 


TASK  1081  Perform  nonprecision  approach 

GRADE:  S+  S  S-  U 

Basic  Qualities:  _ ,  _ 

NOTES : 

-  Inbound  leg  properly  timed  ?  Yes/No  (Circle  one) 

-  Aircraft  in  position  to  safely  land?  Yes/No  (Circle  one) 

Figure  4.  Scenario  worksheet. 
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Interview  guides  used  in  the  experiment  to  measure  crew 
familiarity  consisted  of  two  parts.  First,  each  individual  was 
asked  to  rate  his  performance  as  a  member  during  the  data 
collection  missions.  A  questionnaire  was  used  to  collect  crew 
members'  perceptions  of  how  well  they  performed  the  crew 
coordination  Basic  Qualities  in  terms  of  effectiveness, 
difficulty,  and  style  during  their  battle-rostered  and  mixed  crew 
missions.  Table  4  shows  the  crew  member  self-rating  areas, 
definitions  for  each  area,  and  the  rating  scale.  An  example  of 
the  crew  member  self-rating  questionnaire  is  presented  in 
Appendix  A.  Second,  crew  members  were  asked  in  a  group  forum  to 
respond  to  open-ended  questions  about  crew  coordination  training 
and  battle  rostering  policy. 


Table  4 

Crew  Member  Performance  Self-rating  Areas 


Crew  members 

(n=23 ) 

* 

Self-rating 

area 

Description 

Rating  scale 

Ef fectiveness 

Own  view  as  to  how  well  you  performed 
this  aspect  of  crew  coordination  as  a 
crew 

1  =  Very  poor 

7  =  Superior 

Difficulty 

Own  view  as  to  the  level  of  difficulty 
you  encountered  in  performing  this 
aspect  of  crew  coordination 

ia  =  Very  easy 

7a  =  Very 

Difficult 

Style 

Own  view  as  to  the  manner  in  which  you 
performed  this  aspect  of  crew 
coordination;  that  is,  how  explicitly 
were  crew  coordination  actions  carried 
out  during  the  missions 

la  =  Consistently 
explicit 

7a  =  Consistently 
implicit 

Explicit 

style 

Crew  coordination  was  conducted  in  an 
open  and  observable  manner;  nothing 
was  taken  for  granted;  crew 
coordination  was  accomplished  "by  the 
book" 

Implicit 

style 

Crew  coordination  was  conducted  in  a 
nonobservable  and  subconscious  manner; 
other  crew  member's  understanding  was 
assumed  to  be  consistent  with  yours; 
relied  on  mutual  anticipation  based  on 
previous  flight  experiences 

The  difficulty  and  style  ratings  were  translated  for  computer  analysis  so 
that  1  =  Difficult  and  7  =  Easy;  and  1  =  Implicit  and  7  =  Explicit, 
respectively. 
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Procedure 


Training.  Procedures  to  prepare  for  and  execute  the 
research  included  training  and  data  collection  activities.  Major 
activities  for  both  the  training  and  experiment  phases  of  the 
research  are  presented  in  Table  5. 

The  first  phase  of  the  research  was  conducted  from  May  to 
July  1993  to  provide  all  participating  crew  members  with  crew 
coordination  training.  During  this  phase,  project  staff 
instructed  DES  SIPs  to  assist  in  evaluating  the  unit  instructor 
evaluation  and  training  missions.  Project  staff  and  DES  then 
instructed  and  evaluated  the  participating  IPs  and  unit  trainers 
(UT) .  Instructor  teams  consisting  of  unit  IPs  and  UTs  were 
formed  to  team-teach  the  unit  aviators.  All  participating  AH-64 
aircrews  were  certified  as  having  successfully  completed  the 
Army's  Aircrew  Coordination  Exportable  Training  Package  (Pawlik 
et  al. ,  1992)  . 


Table  5 

Research  Activities 


Training  phase 
(May  to  July  1993) 

Experiment  phase 
(October  to  November  1993) 

Evaluator  training 
•  DES  SIPs 

Data  collection  team  training 

•  IP  evaluators 

•  CMS  operators 

Instructor  training 

•  Project  staff 

•  IPs  and  UTs 

Aircrew  Training3 

•  AH-64  crews 

•  OH-58  crews 

•  UH-60  crews 

Experiment  missions  flown 

•  AH-64  battle  rostered  crews 

•  AH-64  mixed  crews 

Participant  exit  interviews 
(Training) a 

d  t  i _ i  i  it  t- 

Participant  .exit  interviews 
(Experiment) 

d Includes  battalion  staff  personnel. 
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Several  months  elapsed  between  the  training  and  the 
experimental  evaluation.  During  this  time,  subject  aircrews  were 
able  to  refine  their  crew  coordination  skills  and  to  fly  as 
battle-rostered  crews.  No  structured  process  was  followed  nor 
required. 

The  experiment  phase  of  the  research  was  initiated  in  late 
October  1993  by  training  a  fully  integrated  data  collection  team 
of  IP  evaluators,  CMS  operators,  and  project  staff  for  the 
experiment.  Key  project  staff  instructed  data  collection  team 
members  to  provide  them  with  a  thorough  understanding  of  the  data 
collection  plan,  procedures,  and  materials  and  the  knowledge 
necessary  to  evaluate  crew  performance  and  collect  performance 
data.  The  instruction  included  classroom  discussion  and  practice 
exercises  on  the  data  collection  measures,  methods,  and  data 
collector  responsibilities.  Data  collection  team  training 
concluded  with  a  full  dress  rehearsal  of  all  four  data  collection 
scenarios  to  evaluate  and  critique  data  collection  procedures  and 
data  collection  team  performance. 

Data  collection.  A  data  collection  plan  was  developed  and 
implemented  to  provide  a  combination  of  data  collection  means  to 
ensure  reliable  data  for  each  measurement  area.  Most  of  the  data 
collection  methods  had  been  used  successfully  in  previous  efforts 
related  to  the  aircrew  coordination  training  validation  testbed 
(Simon  &  Grubb,  in  preparation) .  Thus,  debugging  the  data 
collection  methods  required  minimal  effort,  and  it  was  known  that 
the  methods  were  valid.  Data  collection  procedures  produced  a 
cross-check  among  collection  sources  without  burdening  any  one 
source  or  requiring  extensive  review  of  mission  videotapes. 

Table  6  shows  the  collection  source  to  measurement  area 
relationships . 

The  data  collected  for  each  mission  in  the  experiment 
included  video  recording,  live  observation,  CMS-generated 
documentation,  and  post-mission  interviews.  Three  videotapes 
were  used  to  document  all  aspects  of  each  mission.  One  tape  was 
used  to  record  the  crew's  premission  planning  and  rehearsal  and 
the  crew-level  after-action  review.  Two  separate  but 
simultaneously  recorded  videotapes  were  used  to  record  the  flight 
in  the  CMS.  One  flight  videotape  was  used  by  project  staff 
observers  in  conjunction  with  data  logger  entries  and  by  IP 
evaluators  to  debrief  each  crew.  Project  staff  and  IP  evaluators 
used  the  second  flight  videotape  as  a  backup  in  the  event  of 
defects  in  the  primary  videotape. 

Live  observation  data  were  collected  by  IP  evaluators,  CMS 
operators,  and  project  staff.  IP  evaluators  used  scenario 
worksheets  and  grade  slips  to  record  data  for  all  measurement 
areas.  The  CMS  operator  controlled  the  preprogrammed  scenario 
for  each  data  collection  mission.  In  addition  to  adding  realism 
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Table  6 


Data  Collection 

Sources  and 

Measures 

Measure 

Collected  by 

IP 

evaluator 

CMS 

Project 

staff 

Partici¬ 

pant 

Basic 

Qualities 

X 

X 

ATM  Tasks 

X 

Mission 

performance 

X 

X 

X 

Familiarity 

effects 

X 

by  role  playing  other  aircraft  in  the  flight  and  external 
tactical  elements,  the  CMS  operator  directed  preselected  display 
screens  to  a  printer  to  collect  specific  mission  performance 
data.  Project  staff  recorded  data  and  comments  corresponding  to 
the  flight  videotape  for  each  mission. 

The  data  collection  team  evaluated  the  subject  aircrews  as 
they  completed  each  of  the  four  scenarios  to  produce  48  data 
collection  missions.  Subject  aircrews  conducted  two  missions  in 
a  battle-rostered  crew  and  two  missions  in  a  mixed  crew 
configuration. 

Exit  interviews .  When  all  of  the  data  collection  missions 
were  completed  for  the  experiment,  crew  members  and  IP  evaluators 
participated  in  an  exit  interview.  Crew  members  were  asked  to 
self-rate  their  performance  during  the  data  collection  missions. 
Crew  members  and  IP  evaluators  were  asked  to  respond  to  open- 
ended  questions  about  crew  coordination  training  and  battle¬ 
rostering  effects  and  policies.  Several  days  prior  to  the  exit 
interview,  participants  were  given  the  open-ended  interview 
questions  to  make  notes  on  the  items  they  wanted  to  discuss.  IP 
evaluators  and  many  of  the  crew  members  wrote  responses  to  the 
interview  questions  and  gave  them  to  the  project  staff.  Three 
groups  of  eight  crew  members  and  one  group  of  seven  IP  evaluators 
and  CMS  operators  were  interviewed.  Each  group  exit  interview 
lasted  approximately  50  minutes. 

The  project  staff  facilitated  the  exit  interview  process  by 
recording  the  group's  responses  and  discussions.  After  the  crew 
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member  self-ratings  plus  participant  written  responses  and 
project  staff  notes  were  collected,  self-rating  data  were 
analyzed  for  insights  into  crew  performance.  Data  from  the  open- 
ended  questions  were  edited  for  readability  and  compiled  to 
eliminate  duplicate  responses.  Summaries  of  responses  to  open- 
ended  questions  are  presented  in  Appendix  B. 

Variables 


Independent  variable.  Crew  condition,  that  is,  battle 
rostered  or  mixed,  was  the  independent  variable  in  this 
experiment.  To  control  the  crew  condition  variable  throughout 
the  experiment,  each  subject  crew  was  given  a  battle-rostered 
crew  number  from  1  to  12.  During  the  experiment,  each  battle- 
rostered  crew  flew  two  evaluation  missions  with  their  battle- 
rostered  crew  member,  i.e.,  24  battle  rostered  crew  missions. 
Copilot  gunners  were  randomly  paired  with  pilots  within  each 
company  to  form  mixed  crews,  i.e.,  mixed  crew  condition.  Each 
mixed  crew  was  given  a  crew  number  from  Ml  to  M24  and  flew  one 
evaluation  mission  during  the  experiment  or  24  mixed  crew 
missions.  Table  7  shows  the  distribution  of  subject  aircrews  by 
attack  helicopter  company. 


Table  7 

Subject  Aircrews  by  Attack  Helicopter  Company 


Attack  helicopter 
company 

Battle-rostered 

crews 

Mixed  crews 

A 

1-4 

Ml  -  M8 

B 

Cn 

1 

00 

M9  -  Ml  6 

C 

9-12 

M17  -  M2 4 

Totals 

12a 

24 

aEach  battle-rostered  crew  completed  two  evaluation  missions. 


To  avoid  potential  bias,  data  collection  missions  were 
identified  by  mission  numbers  1  through  48  rather  than  by  crew 
number.  Also,  IP  evaluators  were  scheduled -to  avoid  evaluating 
the  same  crew  twice. 

Dependent  variables.  The  dependent  variables  for  this 
experiment  were  crew  behaviors  (Basic  Qualities),  ATM  task 
performance,  mission  performance,  and  negative  familiarity 
effects  (Table  8) . 
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Results 


Results  are  presented  with  respect  to  differences  in  the 
crew  condition  independent  variable  across  the  dependent  variable 
measures  of  crew  performance.  Data  are  reported  from  a  differences 
perspective  for  each  performance  measure.  All  statistical  analyses 
were  completed  using  the  Statistical  Package  for  the  Social 
Sciences  Version  6.0  (SPSS  for  Windows;  Norusis,  1993). 

Behaviors  (Basic  Qualities) 

Basic  Quality  ratings  were  made  by  IP  evaluators  during  each 
data  collection  mission.  On  average,  crew  performance  of  the  13 
crew  coordination  Basic  Qualities  produced  mixed  results  with  no 
statistically  significant  difference  in  IP  evaluator  ratings 
between  battle-rostered  and  mixed  crews.  Table  9  compares  Basic 
Quality  item  means  between  battle-rostered  and  mixed  crew 
compositions.  Mean  ratings  for  every  Basic  Quality  were  at  or 
above  the  acceptable  level  rating  of  4.  Battle-rostered  crews 
performed  slightly  higher  than  mixed  crews  on  nine  Basic  Qualities, 
whereas  mixed  crews  performed  slightly  higher  on  four  Basic 
Qualities . 

Ratings  were  compared  using  a  paired  t-test  to  determine 
whether  the  Basic  Quality  performance  differences  were  significant. 
In  no  case  was  the  difference  in  performance  of  crew  coordination 
Basic  Qualities  statistically  significant.  As  summarized  in  Table 
9,  the  mean  rating  for  all  13  Basic  Qualities  was  almost  identical 
for  battle-rostered  and  mixed  crews. 

Task  Performance 


IP  evaluators  graded  crews  on  25  selected  ATM  tasks  for  each 
data  collection  mission.  The  25  tasks  were  selected  for  the 
following  reasons: 

1.  They  provided  a  common  means  for  comparing  battle- 
rostered  and  mixed  crew  evaluation  missions. 

2.  They  are  crew  coordination  intensive. 

3.  They  were  explicitly  presented  in  the  evaluation 
scenarios . 

The  ATM  task  grades  were  converted  for  computer  analysis  so 
that  S+=3,  S=2,  S-=l,  and  U  =  0.  Grades  were  compared  using 
a  paired  t-test  to  determine  whether  task  performance  differences 
were  significant. 
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Table  9 


Behaviors  (Basic  Qualities)  Comparisons  Between  Mean  Rating  Scores 
for  Battle-Rostered  and  Mixed  Crews 


Crews  (n=24) 


Basic  Quality 

Battle 

rostered 

Mixed 

1 

Establish  and  maintain  flight  team 
leadership  and  crew  climate 

5.33 

5.25 

2 

Premission  planning  and  rehearsal 

5.17 

5.33 

3 

Selection  of  appropriate  decision¬ 
making  techniques 

4.46 

4.58 

4 

Prioritize  actions  and  distribute 
workload 

4.63 

4.54 

5 

Management  of  unexpected  events 

4.50 

4.38 

6 

Statements  and  directives  clear, 
timely,  relevant,  complete,  and 
verified 

4.75 

4.83 

7 

Maintenance  of  mission  situational 
awareness 

4.21 

4.04 

8 

Decisions  and  actions  communicated 
and  acknowledged 

4.50 

4.38 

9 

Supporting  information  and  actions 
sought  from  crew 

4.42 

4.33 

10 

Crew  member  actions  mutually  cross- 
monitored 

4.50 

4.17 

11 

Supporting  information  and  actions 
offered  by  crew 

4.54 

4.58 

12 

Advocacy  and  assertion  practiced 

4.54 

4.38 

13 

Crew-level  after-action  reviews 
accomplished 

5.04 

4.88 

All  13  Basic  Quality  Ratings 

4.66 

4.59 

On  the  average,  crew  performance  of  the  25  AH-64  ATM  tasks 
produced  mixed  results  with  no  statistically  significant  difference 
in  IP  evaluator  grades  between  battle-rostered  and  mixed  crews. 
Table  10  compares  each  of  the  25  ATM  task  grades  with  the  average 
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grade  for  all  25  tasks.  The  table  also  compares  the  overall  grade 
for  battle-rostered  and  mixed  crew  missions.  Mean  grades  for  every 
ATM  task  approached  or  exceeded  the  satisfactory  level  of 
performance.  Battle-rostered  crews  performed  slightly  higher  than 
mixed  crews  on  10  ATM  tasks,  whereas  mixed  crews  performed  slightly 
higher  on  9  ATM  tasks.  Performance  of  six  ATM  tasks  was  the  same 
for  both  crew  compositions.  In  no  case  was  the  difference  in 
performance  of  ATM  tasks  statistically  significant.  As  summarized 
in  Table  10,  the  mean  grade  for  all  25  ATM  tasks  was  almost 
identical  for  battle-rostered  and  mixed  crews.  The  overall  mission 
grade  for  both  crew  compositions  was  the  same. 

More  precise  information  is  available  by  linking  ATM  task 
performance  and  crew  behaviors.  When  an  ATM  task  was  graded  S-  or 
U  and  the  less  than  satisfactory  grade  involved  crew  coordination, 
IP  evaluators  noted  the  Basic  Qualities  contributing  to  the  grade. 
Table  11  shows  which  ATM  tasks  were  graded  less  than  satisfactory 
and  which  Basic  Qualities  inhibited  satisfactory  performance  for 
battle-rostered  and  mixed  crews.  Table  entries  and  totals  are  the 
number  of  times  an  IP  evaluator  determined  that  the  Basic  Quality 
contributed  to  a  less  than  satisfactory  grade  for  the  ATM  task. 
Within  each  column,  numbers  to  the  left  of  the  vertical  line 
pertain  to  battle-rostered  crews  and  numbers  to  the  right  are  mixed 
crews . 


Looking  across  the  rows  in  Table  11,  one  can  see  that  crews 
did  not  perform  all  ATM  tasks  satisfactorily  for  all  missions. 

ATM  tasks  1033  "Terrain  Flight  Mission  Planning"  and  2050  "Select 
Appropriate  Weapon  System"  are  exceptions  in  that  they  were  not 
graded  unsatisfactory  due  to  crew  coordination  errors.  The 
following  is  a  list  of  the  ATM  tasks  that  caused  the  most  problems, 
defined  as  those  tasks  with  10  or  more  Basic  Quality  negative 
notations : 

1023  Fuel  management  procedures 
1035  Terrain  flight 
1064  Terrain  flight  navigation 
1081  Nonprecision  approach 
1119  Firing  position  operations 
1140  Engage  target  with  Hellfire 
2008  Evasive  maneuvers 

Battle-rostered  crews  were  graded  unsatisfactory  due  to  crew 
coordination  errors  more  often  than  mixed  crews  on  ATM  tasks  1016 
"Hover  Power  Check"  and  1064  "Terrain  Flight  Navigation." 
Conversely,  mixed  crews  received  unsatisfactory  grades  due  to  crew 
coordination  errors  on  ATM  tasks  1023  "Fuel  Management  Procedures" 
and  1035  "Terrain  Flight." 
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Table  10 


Task 

Grade  Comparisons  Between  Battle- 

-Rostered 

and  Mixed 

Crews 

(n=24 ) 

AH-64  ATM  Task 

Battle 

rostered 

Mixed 

1000 

Crew  Mission  Briefing 

2.29 

2.38 

1004 

DA  Form  5701-R  (PC) 

2.00 

2.00 

1007 

Engine-start,  run-up,  hover,  and 
before  take-off  checks 

1.95 

2.00 

1016 

Hover  Power  Check 

1.88 

2.04 

1023 

Fuel  Management  Procedures 

1.96 

1.67 

1026 

Doppler  Navigation 

1.79 

1.92 

1033 

Terrain  Flight  Mission  Planning 

2.30 

2.50 

1034 

Terrain  Flight  Takeoff 

2.00 

1.96 

1035 

Terrain  Flight 

1.92 

1.58 

1038 

Terrain  Flight  Approach 

2.00 

1.83 

1064 

Terrain  Flight  Navigation 

1.82 

2.10 

1068 

Emergency  Procedures 

1.92 

1.92 

1076 

Radio  Navigation 

1.92 

1.75 

1081 

Nonprecision  Approach 

1.75 

1.67 

1083 

IMC  Procedures/VHIRP 

2.08 

2.00 

1090 

Masking  &  Unmasking 

2.00 

1. 96 

1095 

A/C  Survivability  Equipment 

2.13 

1.96 

1119 

Firing  Position  Ops 

1.79 

1.88 

1140 

Engage  Tgt  w/Hellfire 

1.92 

1.96 

1141 

Engage  Tgt  w/ARCS 

1.93 

1.79 

1142 

Engage  Tgt  w/AWS 

1.85 

1.89 

2008 

Evasive  Maneuvers 

1.87 

1.79 

2043 

FARP  Procedures 

1.94 

2.00 

2050 

Select  Appropriate  Weapon  System 

2.00 

2.04 

2052 

ID  Targets  w/TADS 

1.92 

1.88 

Average  Grade  for  the  25  Tasks 

1.96 

1.95 

Overall  Grade  for  Mission 

1.58 

1.50 
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Looking  down  the  columns  in  Table  11,  it  is  apparent  that  all 
of  the  Basic  Qualities  except  for  BQ  13  "Crew-Level  After-Action 
Reviews  Accomplished"  affected  ATM  task  performance  in  varying 
degrees.  The  following  is  a  list  of  the  Basic  Qualities  that 
contributed  the  most  to  unsatisfactory  ATM  task  performance,  as 
defined  as  those  Basic  Qualities  with  10  or  more  total  entries  for 
both  battle-rostered  and  mixed  crews: 

BQ4  Prioritize  actions  and  distribute  workload 

BQ6  Statements  and  directives  clear,  timely,  relevant,  complete, 
and  verified 

BQ7  Maintenance  of  mission  situational  awarenes 

BQ10  Crew  member  actions  mutually  cross-monitored 

BQ11  Supporting  information  and  actions  offered  by  crew 

Collectively,  these  five  Basic  Qualities  accounted  for  137 
(80%)  of  the  negative  notations.  Battle-rostered  crews  received 
more  unsatisfactory  ATM  task  grades  due  to  crew  coordination  errors 
on  two  Basic  Qualities,  whereas  mixed  crews  were  graded 
unsatisfactory  on  ATM  tasks  more  often  than  were  battle-rostered 
crews  on  three  of  the  five  Basic  Qualities.  Table  12  focuses  on 
these  five  Basic  Qualities  and  compares  the  frequency  of  crew 
coordination  errors  associated  with  less  than  satisfactory  ATM  task 
performance  for  battle-rostered  and  mixed  crews. 

A  statistical  test  of  independent  proportions  was  conducted  to 
determine  if  the  difference  between  battle-rostered  and  mixed  crew 
frequencies  was  significant.  Although  the  sample  is  small,  the 
differences  were  statistically  significant  for  Basic  Quality  6  (z  = 
2.34,  p  =  0.02)  and  Basic  Quality  11  (z  =  -2.12,  p  =  0.03)  . 

Mission  Performance 


Aircrews  were  given  attack  helicopter  tactical  missions  to 
perform  in  the  AH64CMS .  Data  were  collected  from  four  evaluation 
missions  to  determine  if  mission  performance  was  affected  by 
whether  the  crews  were  battle  rostered  or  mixed.  Mission 
performance  data  were  analyzed  in  the  following  categories: 
navigation,  weapons  employment,  threat,  unexpected  events,  and 
instrument  approach.  Performance  measures  that  evaluated  terrain 
flight  navigation,  weapons  systems  employment,  and  threat  avoidance 
and  evasion  were  recorded  during  all  four  evaluation  missions. 
Performance  measures  that  evaluated  unexpected  events,  including 
aircraft  emergencies  and  minor  malfunctions,  and  instrument 
recovery  were  recorded  during  two  or  more  of  the  evaluation 
missions . 
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Table  12 


Crew  Coordination  Errors  Associated  With  Less  Than  Satisfactory  ATM 


Task 

Performance 

Crews 

(n=24 ) 

Basic  Quality 

Battle 

rostered 

Mixed 

4 

Prioritize  actions  and  distribute 
workload 

4 

6 

6 

Statements  and  directives  clear,  timely, 
relevant,  complete,  and  verified 

12 

3 

7 

Maintenance  of  mission  situational 
awareness 

36 

35 

10 

Crew  member  actions  mutually  cross- 
monitored 

12 

18 

11 

Supporting  information  and  actions 
offered  by  crew 

2  * 

9 

*£  <  .05  based  on  600  rating  opportunities  (24  crews  x  25  ATM  tasks) . 


Navigation.  This  performance  measure  evaluates  the  crew's 
ability  to  remain  within  altitude  and  course  limitations  as  well  as 
to  avoid  inadvertent  obstacle  strikes  and  collisions  with  the 
ground.  The  measure  also  evaluates  the  crew's  ability  to  meet 
timed  arrival  requirements  at  designated  checkpoints. 

Crew  terrain  flight  navigation  performance  produced  mixed 
results  with  no  statistically  significant  difference  between 
battle-rostered  and  mixed  crews.  Overall,  battle-rostered  crews 
had  fewer  altitude  and  course  deviations  and  incurred  fewer 
detrimental  outcomes  than  did  mixed  crews.  From  a  crew 
perspective,  67%  of  battle-rostered  crews  and  50%  of  mixed  crews 
completed  their  missions  without  any  detrimental  outcomes.  A  total 
of  18  crews  performed  36  missions,  which  included  timed  arrival 
requirements  at  a  designated  checkpoint.  As  shown  in  Table  13,  75% 
of  all  crews  met  their  arrival  time  requirement  within  2  minutes 
with  little  variation  between  battle-rostered  and  mixed  crews. 

Weapons  employment.  This  performance  measure  evaluates  the 
crew's  ability  to  accurately  acquire,  identify,  and  engage  targets. 
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Table  13 


Comparison  of  Navigation  Arrival  Times 
Crews  (n=18) 


Arrival  time 

Battle 

rostered 

Mixed 

<1 

Minute 

5 

7 

>1 

<2  Minutes 

8 

7 

>2 

<3  Minutes 

1 

2 

>3 

Minutes 

4 

2 

Crew  weapons  systems  employment  performance  produced  mixed 
results,  with  battle-rostered  crews  hitting  and  killing  more 
targets  than  did  mixed  crews.  As  shown  in  Table  14,  battle-rostered 
crews  acquired  more  of  the  total  available  targets  than  did  mixed 
crews,  whereas  the  mixed  crews  engaged  more  of  the  targets  they 
acquired  than  did  battle-rostered  crews.  Battle-  rostered  crews 
hit  and  killed  more  of  the  targets  engaged  (104  of  147)  than  did 
mixed  crews  (82  of  139)  .  A  test  of  proportions  was  used  to 
determine  whether  the  mission  performance  differences  in  weapons 
employment  were  significant.  The  difference  in  the  number  of 
targets  hit  and  killed,  including  targets  hit  and  killed  by 
autonomous  missile  engagements,  between  battle-rostered  and  mixed 
crews  was  statistically  significant  (z  =  2.08,  £  =  0.04).  However, 
one  battle-rostered  crew  improperly  identified  a  friendly  target 
and  fired  on  it. 

When  each  weapon  system  was  analyzed  separately,  battle- 
rostered  crews  hit  and  killed  more  targets  with  missiles  than  did 
mixed  crews,  including  autonomous  engagements  where  the  difference 
in  hits  and  kills  (57  of  65  versus  46  of  66)  was  found  to  be 
statistically  significant  (z  =  2.51,  p  =  0.02).  The  small 
difference  in  the  proportion  of  targets  hit  and  killed  with  rocket 
and  gun  engagements  by  battle-rostered  and  mixed  crews  was  not 
statistically  significant.  One  mixed  crew  accounted  for  almost 
half  of  the  rocket  hits  and  kills.  The  sum  of  targets  engaged  by 
each  weapon  system  exceeds  the  total  number  of  targets  engaged 
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Table  14 


Comparison  of  Weapons  Systems  Employment  Measures 


Crews  (n=24) 


_ Measure _ 

Number  of  targets  acquired  of 
available  targets 

Number  of  targets  engaged  of 
targets  acquired 


Number  of  targets  hit  and  killed  of 
targets  engaged 

Number  of  targets  hit  and  killed  of 
autonomous  and  remote  missile 
engagements 

Number  of  targets  hit  and  killed  of 
autonomous  missile  engagements 

Number  of  targets  hit  and  killed  of 
rocket  engagements 

Number  of  targets  hit  and  killed  of 

gun  engagements _ 

*£  <  .05  Test  of  proportions. 


Battle 

rostered 

178/232 

147/178 

104/147 

75/80 

57/65 

18/47 

11/55 


Mixed 

165/232 

139/165 

82/139 

60/83 

46/66 

13/33 

9/34 


since  several  crews  engaged  targets  with  multiple  weapon 
systems . 

Threat .  This  performance  measure  involves  the  crew's  ability 
to  avoid  and,  if  detected,  successfully  evade  threat  weapon 
systems.  It  also  describes  certain  detrimental  outcomes  resulting 
from  threat  detection;  that  is,  became  misoriented,  hit  by  a  threat 
system,  or  crashed.  Threat  systems  used  in  all  four  scenarios  were 
representative  of  Soviet-equipped  motorized 'and  armored  units. 

Crew  performance  regarding  threat  avoidance  and  evasion 
produced  mixed  results  with  no  statistically  significant  difference 
between  battle-rostered  and  mixed  crews.  As  shown  in  Table  15, 
mixed  crews  had  fewer  radar  warnings,  fewer  detrimental  outcomes. 
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Table  15 


Comparison  of  Threat  Avoidance  and  Evasion  Measures 
Crews  (n=24) _ 


Measure 

Battle 

rostered 

Mixed 

Number  of  threat  radar  warnings 

34 

31 

Number  of  detrimental  outcomes 

14 

11 

Average  duration  of  radar  warnings 
(seconds) 

34.7 

21.2 

Number  of  crews  completing  missions 
with  no  radar  warnings 

8 

5 

and  an  overall  shorter  average  duration  of  radar  warnings  than  did 
battle-rostered  crews.  Three  battle-rostered  crews  and  three  mixed 
crews  incurred  more  than  one  detrimental  outcome  during  a  mission. 

Because  nonradar-controlled  threat  systems,  that  is,  T80 
tanks,  BMP-2  personnel  carriers,  and  MI-24F  helicopters,  were 
included  in  each  scenario,  several  crews  were  able  to  avoid  radar 
warnings  but  still  incurred  detrimental  outcomes  from  other  threat 
systems.  Battle-rostered  crews  had  better  success  at  avoiding 
radar-controlled  threat  systems  and  were  more  successful  at 
avoiding  all  threat  systems. 

As  shown  in  Table  16,  almost  70%  of  all  the  crews  had  average 
warning  times  less  than  30  seconds  with  more  mixed  crews  than 
battle-rostered  crews  included  in  this  group.  Several  crews 
remained  exposed  to  a  radar-controlled  system  because  they  were 
aware  that  they  were  outside  of  the  weapon  system's  effective 
range.  As  a  result,  these  crews  incurred  unusually  lengthy  warning 
times,  which  adversely  affected  overall  average  warning  time.  For 
example,  3  of  4  battle-rostered  crews  had  average  times  exceeding 
100  seconds.  Warning  times  were  not  recorded  for  one  battle- 
rostered  and  one  mixed  crew. 

Unexpected  events.  This  performance  measure  evaluates  the 
crew's  ability  to  work  in  concert  while  coping  with  emergencies, 
malfunctions,  and  inadvertent  entry  into  instrument  meteorological 
conditions  (IMC) .  Crews  coped  with  a  minor  malfunction  and  an 
inadvertent  entry  into  IMC  during  two  missions,  while  the  other 
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Table  16 


Comparison  of  Average 

Crews  (n=23) 

Radar  Warning 

Times 

Average  radar 
warning  time 

Battle 

rostered 

Mixed 

<15  Seconds 

8 

10 

>15  <30  Seconds 

5 

9 

>30  <60  Seconds 

6 

3 

>60  Seconds 

4 

1 

two  missions  contained  a  minor  malfunction  and  an  aircraft 
emergency.  Crew  performance  during  unexpected  event  situations 
produced  mixed  results  with  no  statistically  significant  difference 
between  battle-rostered  and  mixed  crews. 

The  crew's  ability  to  cope  with  aircraft  emergencies  was 
measured  in  the  second  and  fourth  evaluation  missions.  During  the 
second  mission,  12  crews  experienced  a  loss  of  engine  oil  and 
subsequent  failure  of  the  engine  while  returning  to  the  FARP . 

Every  crew  correctly  diagnosed  the  emergency  and  properly  performed 
the  required  emergency  procedures.  One  battle-rostered  crew 
crashed  performing  the  running  landing  following  this  emergency. 
While  returning  to  the  FARP  during  the  fourth  mission,  12  crews  had 
an  engine  chip  detector  illuminate  followed  15  seconds  later  by  a 
complete  engine  failure.  All  but  one  battle-rostered  crew 
correctly  diagnosed  the  emergency  and  properly  performed  the 
required  emergency  procedures.  One  mixed  crew  crashed  performing 
the  running  landing  following  this  emergency. 

Crew  performance  during  system  malfunctions  was  measured  in 
all  four  missions.  During  the  first  and  third  missions,  crews 
experienced  minor  malfunctions  associated  with  the  electrical  and 
hydraulic  systems,  respectively,  en  route  to  the  FARP.  All  24 
crews  correctly  diagnosed  the  malfunctions.  One  battle-rostered 
and  one  mixed  crew  delayed  initiating  the  emergency  procedures 
until  after  landing,  and  one  battle-rostered  crew  crashed 
performing  the  landing  in  the  FARP. 
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During  the  second  and  fourth  missions,  crews  experienced 
malfunctions  associated  with  the  communications  and  fuel  systems. 
While  occupying  a  battle  position  during  the  second  mission,  both 
FM  radio  circuit  breakers  popped,  temporarily  disabling  both  FM 
radios.  While  en  route  to  the  second  battle  position  during  the 
fourth  mission,  crews  experienced  a  slow  fuel  leak  resulting  in  an 
incremental  loss  of  500  lbs  of  fuel.  No  warnings  are  associated 
with  either  of  these  malfunctions.  The  crews  were  required  to 
detect  these  malfunctions  through  situational  awareness  and  cross¬ 
checks  of  aircraft  systems.  As  shown  in  Table  17,  half  of  all  the 
crews  were  successful  in  detecting  these  more  subtle  malfunctions 
in  less  than  seven  minutes  with  more  mixed  crews  than  battle- 
rostered  crews  included  in  this  group. 


Table  17 

Comparison  of  Malfunction  Detection  Times 
Crews  (n=12)  _ 


Average  time  to 
detect  malfunction 

Battle 

rostered 

Mixed 

<,2  Minutes 

2 

3 

>2  <7  Minutes 

3 

4 

>7  <20  Minutes 

3 

2 

>20  Minutes 

4 

3 

Inadvertent  entry  into  IMC  was  measured  during  the  first  and 
third  missions.  Crews  experienced  deteriorating  weather  conditions 
on  the  flight  back  to  the  FARP,  which  resulted  in  a  loss  of  visual 
reference  while  terrain  flying.  Every  crew  except  one  mixed  crew 
successfully  completed  the  vertical  helicopter  instrument  recovery 
procedure  (VHIRP) ,  transitioning  from  visual  to  instrument 
meteorological  conditions. 

Instrument  approach.  This  performance  measure  evaluates  the 
crew's  ability  to  plan  and  execute  a  non-precision  instrument 
recovery  procedure  after  inadvertently  entering  IMC.  This  measure 
was  recorded  for  12  battle-rostered  and  12  mixed  crews  during  the 
first  and  third  missions.  Crew  instrument  approach  performance 
produced  mixed  results  with  no  statistically  significant  difference 
between  battle-rostered  and  mixed  crews. 


35 


Three  dimensions  of  this  measure  were  observed  for  each  crew: 
planning  the  approach,  timing  the  inbound  leg,  and  the  outcome  of 
the  approach.  Table  18  shows  the  results  for  each  dimension  by 
battle-rostered  and  mixed  crews. 


Table  18 

Comparison  of  Instrument  Approach  Performance 
Crews  (n=12)  ' _ 


Measure 

Battle 

rostered 

Mixed 

Planning  the  approach 

Superior 

1 

2 

Good 

4 

4 

Poor 

7 

6 

Unsatisfactory 

0 

0 

Timing  the  inbound  leg 

10 

6 

Successful  outcome  of  the 
approach 

10 

9 

Most  crews  received  a  poor  rating  for  approach  planning  with 
little  variation  between  battle-rostered  and  mixed  crews.  More 
battle-rostered  crews  than  mixed  crews  properly  timed  their 
approaches.  Most  crews  successfully  completed  the  instrument 
approach  with  little  variation  between  battle-rostered  and  mixed 
crews.  All  of  the  crews  with  superior  ratings  for  approach 
planning  completed  all  requirements  of  the  approach  successfully; 
that  is,  the  crew  properly  planned,  timed,  and  executed  the 
approach. 

Participant  Exit  Interviews 

Crew  members  and  IP  evaluators  were  interviewed  after  all  of 
the  data  collection  missions  were  completed. 

Self-ratings .  Anecdotal  evidence  from  aviation  accident 
investigations  and  informal  comments  made  by  Army  aviators  during 
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this  and  previous  research  suggested  a  performance  downside  to 
battle  rostering.  Specifically/  it  was  suggested  that  repeated 
flights  with  the  same  crew  member  builds  confidence,  which  can  lead 
to  overconfidence  and  complacency.  It  was  also  suggested  that  crew 
members  were  more  formal  and  overt  in  carrying  out  procedures  and 
crew  coordination  actions  when  flying  with  other  than  their  battle- 
rostered  crew.  The  crew  member  self-rating  questionnaire  was 
designed  to  provide  a  subjective  measure  of  these  aspects  of  crew 
coordination  to  supplement  the  behaviors  (Basic  Qualities)  and  task 
performance  data  collected  earlier.  Table  19  shows  the  results  of 
crew  member  self-ratings  on  three  aspects  of  crew  coordination 
performance  in  battle-rostered  and  mixed  crews. 

Crew  member  self-ratings  produced  mixed  results  with 
statistically  significant  differences  between  battle-rostered  and 
mixed  crews.  A  paired  t-test  revealed  that  crew  members  rated 
themselves  overall  more  effective  (5.9  versus  5.4)  when  operating 
in  a  battle-rostered  crew  than  in  a  mixed  crew  configuration 
(t  =  3.69,  df  =  22,  p  =  .001) .  They  rated  their  workload 
difficulty  in  a  battle-rostered  crew  as  easier  (5.6  versus  4.9) 
than  in  a  mixed  crew  (t  =  4.18,  df  =  22,  £  =  .000) .  These  same 
crew  members  rated  their  crew  coordination  style  as  more  implicit 
(4.2  versus  4.9)  in  a  battle-rostered  crew  than  when  in  a  mixed 
crew  (t:  =  -3.07,  df  =  22,  £  =  .006)  . 

Crew  members  completing  the  self-rating  questionnaire 
represented  22  of  the  24  battle-rostered  crews  and  19  of  the  24 
mixed  crews  participating  in  the  experiment.  Both  battle-rostered 
and  mixed  crews  rated  themselves  higher  than  their  actual 
performance  as  measured  by  IP  evaluators.  Table  20  compares  crew 
self-ratings  with  IP  evaluator  ratings  of  the  13  behaviors  (Basic 
Qualities) .  The  amount  that  crews  overrated  their  performance  is 
an  estimate  of  overconfidence.  Battle-rostered  crews  were  50%  more 
overconfident  in  their  performance  than  mixed  crews. 

Crew  member  self-ratings  of  crew  coordination  style  were  used 
to  determine  the  crew  coordination  style  of  the  crew;  that  is, 
implicit  or  explicit  style.  When  either  or  both  crew  members  rated 
their  overall  crew  coordination  performance  as  implicit,  the  crew 
was  identified  as  having  operated  using  an  implicit  crew 
coordination  style.  Table  21  shows  a  comparison  of  implicit 
coordination  styles  between  battle-rostered  and  mixed  crews'. 
Implicit  crew  coordination  style  was  associated  more  frequently 
(64%  versus  42%)  with  battle-rostered  crews  than  with  mixed  crews. 
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Self-ratings  of  Crew  Coordination  Performance 
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Table  20 


Crew  Confidence 


Crew  members  (n=23) 


Rater 

Battle 

rostered 

Mixed 

Crew  self-ratings  of 
13  Basic  Qualities 

5.9 

5.4 

IP  evaluator  ratings 
of  13  Basic 

Qualities 

4.7 

4.6 

Overconfidence 

factor 

1.2 

0.8 

Table  21 

Implicit  Coordination  Style 

Crew  members  (n=23) 

Battle 

rostered 

Mixed 

Number  of  crews 
rated 

22 

19 

Number  of  crews  with 
implicit 

coordination  style 

14 

8 

Percent  implicit 
style 

64% 

42% 

Open-ended  questions.  In  Appendix  B,  the  open-ended  exit 
interview  questions  are  stated,  a  summary  of  the  responses  is 
given,  the  detailed  responses  are  provided,  and  the  project  staff's 
reaction  to  each  set  of  comments  is  stated.  .  Many  of  the  experiment 
conclusions  and  recommendations  were  generated  or  reinforced  during 
the  exit  interviews.  The  reader  is  encouraged  to  closely  examine 
Appendix  B.  Following  is  a  summary  of  crew  member  and  IP  evaluator 
comments : 
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1.  Battle-rostered  crews  tended  to  abbreviate  mission 
planning  with  comments  such  as,  "Like  we  always  do."  When  pressed 
for  details,  crew  members  did  not  have  a  common  understanding.  (IP 
Evaluator) 

2.  Performance  depends  more  on  the  level  of  crew 
experience  than  whether  or  not  they  are  battle  rostered.  (Crew 
member) 

3.  There  is  a  better  mission  flow  with  a  battle-rostered  crew 
member.  [Others  disagreed.]  I  am  more  deliberate  in  a 

mixed  crew.  I  didn't  "shoot  from  the  hip"  as  I  did  with  my 
battle-rostered  crew  member.  (Crew  member) 

4.  Quicker  detection  of  error  chains  with  battle-rostered 
crews  was  purely  a  perception  on  the  part  of  the  crews.  In 
reality,  breaking  error  chains  depends  on  crew  member 
assertiveness.  (IP  Evaluator) 

5.  Some  complacency  is  found  with  battle  rostering.  Flying 
with  other  crews  guards  against  complacency.  You  learn  things  from 
different  aviators.  (Crew  member) 

6.  Confidence  can  lead  to  overconfidence  and  complacency. 

The  one  time  your  battle-rostered  buddy  doesn't  perform  as  expected 
comes  as  a  big  surprise.  (Crew  member) 

7.  Battle-rostered  crews  had  well  established  crew  climates 
and  were  familiar  with  each  other.  (IP  Evaluator) 

8.  There  was  more  cross-monitoring  and  prompting  (providing 
information)  with  the  mixed  crew.  More  information  is  passed  with 
a  mixed  crew.  (Crew  member) 

9.  Coordination  was  better  in  the  mixed  crew,  even  though  we 
performed  adequately  in  both  cases.  (Crew  member) 

10.  Experience  has  a  big  influence  on  your  confidence  level 
and  how  you  operate  as  a  crew  member.  Confidence  can  lead  to 
overconfidence  and  complacency.  (Crew  member) 

11.  Battle-rostered  crews  were  more  informal,  complacent,  and 
overlooked  things.  Mixed  crews  were  more  formal,  attentive,  and 
did  not  overlook  things  as  much.  (IP  Evaluator) 
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12.  Implied  communication  and  implied  tasks  were  more 
prevalent  with  battle-rostered  crews.  (IP  Evaluator) 

Discussion 

This  experiment  provided  little  evidence  that  battle  rostering 
contributes  additional  significant  improvements  to  mission 
performance  or  flight  safety  as  compared  to  the  use  of  crew 
coordination  training  alone.  In  fact,  experimental  evidence 
suggests  that  battle  rostering  induces  a  greater  degree  of 
overconfidence,  which  is  a  possible  contributor  to  aviation 
accidents.  The  experiment  applied  the  crew  coordination  ETP  pre- 
and  post-training  method  and  measures  to  examine  the  marginal 
contributions  of  battle  rostering.  Interpretations  made  by  data 
collectors  and  project  staff  during  the  conduct  of  the  experiment 
are  incorporated  in  this  discussion. 

The  discussion  of  the  results  of  this  experiment  is  organized 
into  three  sections.  The  first  section  examines  mission 
performance  and  addresses  the  crew  behavior,  task  performance,  and 
mission  performance  hypotheses.  The  second  and  third  sections 
discuss  results  relevant  to  flight  safety  and  familiarity  effects, 
respectively. 

Mission  Performance 


The  overall  effect  of  crew  composition  on  crew  coordination 
behavior  and  task  performance  was  insignificant.  Battle-rostered 
and  mixed  crew  ratings  on  behaviors  (Basic  Qualities)  and  grades  on 
ATM  tasks  offered  few  insights  into  performance  differences  except 
for  the  area  of  less  than  satisfactory  task  performance.  The 
analysis  of  crew  coordination  deficiencies  associated  with  less 
than  satisfactory  task  performance  suggested  that  crew  composition 
may  affect  crew  coordination  behaviors  in  the  cockpit.  A 
statistically  significant  difference  was  found  between  battle- 
rostered  and  mixed  crews  for  two  Basic  Qualities. 

Battle-rostered  crews  received  more  unsatisfactory  task  grades 
due  to  errors  in  performing  Basic  Quality  6  "Statements  and 
Directives  Clear,  Timely,  Relevant,  Complete,  and  Verified."  The 
finding  suggests  that  battle-rostered  crews  tended  to  revert  to  an 
implicit  style  of  crew  coordination,  typically  observed  before  crew 
coordination  training,  more  often  than  mixed  crews  did.  Crew 
composition  also  affected  how  mixed  crews  coordinated  in  the 
cockpit.  Mixed  crews  were  graded  unsatisfactory  more  often  than 
battle-rostered  crews  for  errors  in  Basic  Quality  11  "Supporting 
Information  and  Actions  Offered  by  Crew."  Speculation  by  IP 
evaluators  and  project  staff  is  that  crew  members  in  mixed  crews 
tended  to  concentrate  more  on  their  individual  duties  and 
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responsibilities  rather  than  on  supporting  their  fellow  crew  member 
with  information  and  assistance. 

Unsatisfactory  performance  of  specific  tasks  due  to  crew 
coordination  errors  can  also  be  attributed  to  crew  composition 
effects.  The  tendency  of  battle-rostered  crews  to  coordinate 
implicitly  contributed  to  their  less  than  satisfactory  performance 
of  Task  1016  "Hover  Power  Check"  and  Task  1064  "Terrain  Flight 
Navigation."  Both  of  these  tasks  require  explicit  call  out  and 
response  items  to  achieve  the  performance  standard.  Less  than 
satisfactory  performance  of  Task  1035  "Terrain  Flight"  by  mixed 
crews  is  believed  to  be  partially  attributable  to  failure  to 
provide  information  and  assistance/  an  effect  of  crew  composition. 
Including  information  in  crew  coordination  training  on  the  nature 
of  these  tendencies  when  operating  in  a  battle-rostered  or  mixed 
crew  is  a  possible  remedy  for  less  than  satisfactory  task 
performance . 

In  comparing  battle-rostered  versus  mixed  crews  across  the 
five  areas  of  mission  performance  measured  in  the  experiment,  only 
one  difference  was  found  to  be  statistically  significant.  Battle- 
rostered  crews  achieved  more  missile  kills  per  target  engaged. 
Engaging  targets  with  missiles  from  the  AH-64  requires  precise 
target  acquisition  and  weapon  systems  settings,  adjustments,  and 
sequencing  actions  between  the  pilot  and  CPG.  Familiarity  with  the 
other  crew  member's  experience  and  preferred  techniques  provides  a 
time  advantage  in  anticipating  and  offering  information  and 
assistance.  This  advantage  can  result  in  fewer  lost  targets  and 
lost  missiles.  These  results  indicate  that  battle  rostering 
aircrews  has  a  potential  positive  effect  on  gunnery  performance. 

Flight  Safety 

Crew  composition  had  no  significant  effect  on  flight  safety. 
Crew  flight  safety  was  evaluated  using  the  results  from  the 
experiment's  safety  intensive  task  and  mission  performance 
measures . 

Task  results  were  drawn  from  crew  performance  of  ATM  tasks 
most  frequently  cited  in  aircraft  accident  reports.  These  tasks 
included  terrain  flight,  firing  position  operations,  evasive 
maneuvers,  emergency  procedures,  and  execution  of  a  nonprecision 
instrument  approach.  Remaining  clear  of  obstacles  and  the  ground 
while  performing  these  tasks  requires  crew  members  to  interact  and 
coordinate  actions.  There  was  no  significant  difference  between 
battle-rostered  and  mixed  crew  performance  of  flight  safety  tasks. 

Three  mission  performance  measurement  areas  provided  flight 
safety  results  in  terms  of  detrimental  outcomes.  Detrimental 
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outcomes  for  the  terrain  flight  mission  performance  measure 
included  obstacle  strikes  and  collision  with  the  ground.  The 
additional  measures  of  misorientation  and  recovery  fire  were  used 
to  evaluate  the  threat  evasion  mission.  Unexpected  events 
performance  measures  provided  additional  collision  with  the  ground 
results.  There  was  no  significant  difference  between  battle- 
rostered  and  mixed  crew  flight  safety  mission  performance. 

In  addition  to  task  and  mission  performance  measures,  crew 
flight  safety  was  evaluated  from  an  error  chain  perspective.  IP 
evaluators,  CMS  operators,  and  project  staff  recorded  instances  of 
a  series  of  errors  or  events  with  potential  to  jeopardize  mission 
effectiveness  or  flight  safety.  For  example,  a  crew  recovering 
from  evading  a  threat  system  failed  to  recognize  an  engine  governor 
malfunction,  failed  to  maintain  minimum  safe  flight  parameters,  and 
crashed  while  occupying  a  new  position.  Error  chain  results 
included  crews  that  interacted  to  break  error  chains  early  before 
suffering  adverse  mission  or  safety  consequences.  Both  battle- 
rostered  and  mixed  crews  demonstrated  similar  error  chains. 

Familiarity  Effects 

The  familiarity  effect  results  of  this  experiment  suggest  that 
battle  rostering  crews  has  a  potential  adverse  effect  on  mission 
performance  and  flight  safety.  Results  from  the  crew  member  self¬ 
rating  questionnaire  support  the  hypothesis  that  crew 
overconfidence  and  complacency  are  significantly  higher  with 
battle-rostered  crews. 

The  finding  that  crew  members  rate  themselves  50%  more 
confident  when  flying  in  their  battle-rostered  crew  than  in  a  mixed 
crew  has  negative  implications.  Overconfidence  is  suspected  as  a 
possible  contributor  to  aviation  accidents  and  crew  complacency. 
Battle-rostered  crews  rated  their  workload  as  easier  and  their  crew 
coordination  style  as  more  implicit  than  mixed  crews  did.  This 
last  finding  tends  to  support  the  perception  that  battle  rostering 
fosters  implicitness,  which  can  lead  to  informality  and 
complacency.  Crew  member  perceptions  that  they  detected  and  broke 
more  error  chains  quicker  when  in  a  battle-rostered  crew  were  not 
supported  by  IP  evaluator  observations. 

Including  information  in  crew  coordination  training  on  the 
negative  familiarity  effects  of  battle  rostering  is  a  possible 
hedge  against  mission  performance  and  flight  safety  risk. 

Conclusions  and  Recommendations 

The  crew  composition  experiment  was  successful  in  evaluating 
the  interactive  effects  of  crew  coordination  training  and  battle 
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rostering  on  AH-64  attack  helicopter  battalion  aircrews.  As  shown 
in  Table  22,  crew  coordination  training  administered  to  battle- 
rostered  crews  in  June  1993  significantly  improved  crew  performance 
in  all  measurement  areas.  This  experiment  to  evaluate  the  effects 
of  battle  rostering  on  crew  performance  extended  the  research  using 
the  same  population  of  battle-  rostered  crews  now  fully  trained  in 
the  Army's  crew  coordination  principles.  Both  crew  coordination 
behaviors  (Basic  Qualities)  and  task  performance  remained 
relatively  stable  after  unit  aviators  received  the  Aircrew 
Coordination  Exportable  Training  Package  (Pawlik  et  al.,  1992). 

The  experiment  results  supported  one  of  the  five  research 
hypotheses.  Preliminary  results  provided  sufficient  evidence  to 
support  a  policy  decision  that  makes  battle  rostering  of  aircrews 
optional  rather  than  mandatory.  The  policy  announcement  emphasized 
the  risks  associated  with  battle  rostering  as  identified  in  the 
experiment . 

Conclusions 


The  principal  research  issue  was:  Does  battle  rostering 
produce  observable  improvements  in  mission  performance  and  flight 
safety?  The  overall  conclusion  from  the  experiment  was,  that  while 
some  aviators  perceive  battle  rostering  to  improve  crew 
performance,  minimal  experimental  evidence  shows  meaningful 
improvements  in  mission  performance  or  flight  safety.  In  fact,  the 
contributions  of  battle  rostering  to  crew  mission  effectiveness  and 
flight  safety  were  largely  offset  by  unmet  expectations  and 
drawbacks.  Objective  data  and  participant  comments  concluded  that 
the  principal  contributions  of  battle  rostering  were  as  follows: 

1.  Improved  weapons  employment  mission  performance  (Table 
14) ,  and 

2.  Increased  familiarization  and  confidence  within  crews 
(Table  19) . 

IP  evaluator  observations  and  objective  mission  data  did  not 
confirm  the  intuitive  expectations  of  battle  rostering.  Battle 
rostering  provided: 

1.  No  improvement  in  overall  crew  coordination  performance 
(Table  9) , 

2.  No  improvement  in  overall  ATM  task  performance  (Table  10), 

and 

3.  No  insurance  of  reduced  risk  (Tables  9  and  18)  . 
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Table  22 


Both  objective  and  subjective  data  collected  during  the 
experiment  identified  the  following  operational  and  administrative 
drawbacks  associated  with  battle  rostering: 

1.  Fostered  overconfidence  (Table  20), 

2.  Fostered  informal  and  nonstandard  procedures  (IP  Evaluator 
Open-ended  Questionnaire,  item  7) ,  and 

3.  Fostered  implicit  coordination  behaviors  (Table  21). 

Summary  conclusions  of  this  study  support  one  of  the  five 
research  hypotheses. 

1.  Crew  coordination  behavior  will  be  significantly  improved 
through  a  combination  of  crew  coordination  training  and  battle 
rostering,  as  compared  to  the  use  of  crew  coordination  training 
alone.  Crew  performance  of  the  13  crew  coordination  behaviors 
(Basic  Qualities)  produced  mixed  results  with  no  statistically 
significant  difference  between  battle-rostered  and  mixed  crews. 
Results  from  task  performance  and  crew  member  self-ratings  offered 
sufficient  evidence  to  suggest  that  battle-rostered  crews  tend  to 
revert  to  an  implicit  style  of  crew  coordination  more  often  than 
mixed  crews  do. 

2.  Crew  flight  task  performance  will  be  significantly 
improved  through  a  combination  of  crew  coordination  training  and 
battle  rostering ,  as  compared  to  the  use  of  crew  coordination 
training  alone.  Crew  performance  of  the  25  ATM  tasks  produced 
mixed  results  with  no  statistically  significant  difference  between 
battle-rostered  and  mixed  crews.  The  implicit  crew  coordination 
tendency  demonstrated  by  battle-rostered  crews  produced  an  adverse 
effect  on  their  performance  of  some  ATM  tasks. 

3.  Mission  performance  will  be  significantly  improved  through 
a  combination  of  crew  coordination  training  and  battle  rostering, 
as  compared  to  the  use  of  crew  coordination  training  alone.  While 
there  were  small  but  statistically  significant  differences  in  the 
area  of  weapons  employment,  crew  performance  of  the  five  areas  of 
mission  performance  produced  mixed  results  with  no  other 
statistically  significant  difference.  Battle  rostering  crews  has  a 
potential  positive  effect  on  gunnery  performance. 

4.  Flight  safety  will  be  improved  through  a  combination  of 
crew  coordination  training  and  battle  rostering,  as  compared  to  the 
use  of  crew  coordination  training  alone.  Crew  performance  of  the 
three  flight  safety  intensive  mission  performance  areas  and  the 
safety-related  ATM  tasks  produced  mixed  results  with  no 
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statistically  significant  difference  between  battle-rostered  and 
mixed  crews.  Both  crew  composition  types  demonstrated  similar 
error  chains. 

5.  Crew  overconfidence  and  complacency  will  be  significantly 
higher  with  battle-rostered  crews ,  as  compared  to  non-battle 
rostered  crews.  Battle-rostered  crews  overrated  their  performance 
50%  more  than  did  mixed  crews.  Crew  members  and  IP  evaluators 
commented  that  overconf idence  can  lead  to  complacency.  Crew 
members  rated  their  crew  coordination  style  as  more  implicit  when 
in  a  battle-rostered  crew  than  when  in  a  mixed  crew.  Implicit  crew 
coordination  coupled  with  overconfidence  is  a  potentially 
detrimental  combination  with  adverse  effects  on  mission  performance 
and  flight  safety. 

Recommendations 


The  principal  operational  issue  was,  "Should  Army  policy 
require  commanders  to  battle  roster  aircrews?"  The  Army's  initial 
strategy  was  to  achieve  crew  coordination  by  simultaneously 
mandating  battle-rostered  crews  and  conducting  crew  coordination 
training.  The  results  of  the  research  program  lead  to  the 
conclusion  that  the  crew  coordination  training  component  of  that 
strategy  is  achieving  the  objective.  The  mechanism  for  improving 
crew  coordination,  the  Aircrew  Coordination  Exportable  Training 
Package,  (Pawlik  et  al.,  1992)  is  already  being  fielded  by  the 
USAAVNC ' s  Aviation  Training  Brigade.  The  research  conclusions 
further  suggest  that  crew  coordination  training  and  evaluation  does 
not  have  to  be  tied  to  battle-rostered  crews. 

Following  is  a  two-part  list  of  recommendations.  The  first 
set  of  recommendations  is  implementing  actions  for  this  research  to 
improve  mission  effectiveness  and  flight  safety.  The  second  set  of 
recommendations  is  for  conducting  research  to  address  unresolved 
crew  and  team  coordination  issues. 

Recommendations  -  Implement  current  research. 

1.  Document  the  requirement  that  all  crew  members  (rated  and 
nonrated)  must  receive  crew  coordination  training.  Because 
standardized  training  was  found  to  be  the  preferred  solution  to 
improve  crew  coordination,  it  is  now  more  important  than  ever  to 
make  sure  the  training  is  fully  implemented.  The  requirement  to 
complete  the  Army's  Aircrew  Coordination  Training  Program  (Pawlik 
et  al.,  1992)  should  be  promulgated  in  key  policy  documents,  for 
example,  AR  95-1  (Department  of  the  Army,  1990)  and  TC  1-210 
(Department  of  the  Army,  1992b) .  Emphasize  that  crew  coordination 
training  helps  to  offset  any  negative  habits  introduced  by  repeated 
flights  with  the  same  crew.  Crew  familiarity  effects  can  be 
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mitigated  by  emphasizing  the  importance  of  explicit  crew 
coordination  when  flying  in  a  battle-rostered  crew  and  by 
emphasizing  the  importance  of  supporting  other  crew  members  when 
flying  in  a  mixed  crew. 

2.  Allow  unit  commanders  to  battle  roster  crews  as  a  risk 
management  tool  without  the  administrative  requirement  for 
mandatory  crew  readiness  level  progression.  Results  of  the 
experiment  suggest  that  commanders  may  reduce  risk  and  enhance 
areas  of  mission  performance  by  battle  rostering  crews  for  short 
periods  of  time  for  specific  missions,  for  example,  gunnery. 
Commanders  should  be  cautioned  that  making  permanent  crew 
assignments  can  potentially  jeopardize  both  mission  performance  and 
flight  safety.  Requiring  that  all  crew  members  complete  crew 
coordination  training  before  being  selected  and/or  designated  as 
part  of  a  crew,  eliminates  the  need  to  establish  and  manage  levels 
of  crew  readiness.  Crew  designation  and  continuity  should  be  based 
on  the  pilot  in  command  (PC)  rather  than  on  the  full  complement  of 
individual  crew  members.  Once  aviators  successfully  complete  crew 
coordination  training  and  are  certified  by  the  unit  commander,  they 
should  be  ready  to  serve  as  part  of  any  crew. 

3.  Specify  that  crew  coordination  refresher  training  be 
accomplished  on  an  annual  basis  within  each  unit.  Tendencies 
revealed  during  the  experiment  for  crew  members  to  revert  to  an 
implicit  crew  coordination  style,  and  individual  rather  than  crew 
mission  orientation  make  it  important  to  develop  and  implement  crew 
coordination  refresher  training.  As  a  minimum,  refresher  training 
should  include  an  update  of  recent  crew  coordination  related 
accidents,  a  review  of  observed  crew  coordination  performance  in 
the  unit,  and  a  training  mission  in  the  simulator  or  aircraft.  It 
is  especially  important  to  reinforce  the  need  for  explicit  crew 
coordination. 

4 .  Incorporate  crew  coordination  Basic  Qualities  from  the 
Aircrew  Coordination  Exportable  Training  Package  (Pawlik  et  al., 
1992)  into  each  ATM  for  individual  Annual  Proficiency  and  Readiness 
Test  (APART)  evaluations.  Individuals  and  crews  pay  particular 
attention  to  the  areas  of  their  performance  that  are  formally 
evaluated.  As  shown  in  the  experiment,  crew  coordination 
evaluations  do  not  have  to  be  tied  to  battle-rostered  crews.  Unit 
IPs  who  have  completed  the  Crew  Coordination  Instructor  Course  are 
fully  trained  to  rate  crew  coordination  performance  using  either 
the  modified  ATM  grade  slip  (Pawlik  et  al.,  1992)  or  the  Aircrew 
Coordination  Evaluation  (ACE)  Checklist  (Grubb,  Simon,  &  Zeller, 
1992)  . 
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Recommendations  -  Research  unresolved  issues. 


1.  Conduct  additional  research  to  address  the  crew 
composition  effects  on  utility,  cargo,  and  observation  crews 
operating  under  low  visibility  conditions  using  individual  night 
vision  devices  (NVD) .  Experiment  results  are  based  on  crew 
performance  in  the  AH-64  attack  helicopter  with  its  fully 
integrated  target  acquisition  designation  sight  and  pilot  night 
vision  system.  Aircraft  without  shared  access  to  sensor  systems 
and/or  symbology  demand  more  explicit  crew  coordination  because  of 
limited  fields  of  view  and  assigned  scan  sectors.  It  is  not  known 
if  battle  rostering  improves  the  performance  of  crew  coordination- 
trained  cargo  and  utility  crews;  for  example,  accurate  and  safe 
navigation  at  night.  Both  the  CH-47  and  UH-60  visual  flight 
simulators  are  capable  of  video  recording  night  missions  under  NVD 
conditions . 

2.  Pursue  additional  research  to  understand  the  effects  of 
complacency  and  time  intervals  between  flights  by  the  same  crew  on 
performance  and  safety.  Operational  mission  requirements  limit  the 
availability  of  aircrews  to  compressed  periods  of  research  data 
collection.  In  this  experiment,  crews  completed  all  48  evaluation 
missions  in  14  days.  It  is  not  known  to  what  extent  temporal 
factors  affect  battle  rostering-fostered  complacency  nor  is  it 
known  how  many  flights  together  it  takes  as  a  crew  before 
overconfidence  becomes  a  factor.  A  longitudinal  study  of  crew 
coordination-trained  crew  members  is  needed. 

3.  Develop  crew  selection  methods  and  tools  to  assist  units 
in  assigning  crew  members  for  training  and  combat  missions. 
Participants  in  the  experiment  indicated  the  need  to  provide  units 
with  improved  crew  selection  capabilities.  Commanders  have  wide 
latitude  in  distributing  risk  and  crew  member  experience.  Reliable 
instruments  are  available  to  measure  personality  traits,  individual 
skills,  and  crew  coordination  skills  as  inputs  to  a  crew  selection 
automated  method.  Technology  is  available  to  develop  advanced  crew 
selection  methods  software  for  use  on  microcomputers.  An  automated 
crew  selection  tool  should  be  developed  to  help  units 
simultaneously  manage  risk  and  provide  high  levels  of  mission 
accomplishment  by  using  all  available  information  to  select  crews. 

4 .  Extend  crew  coordination  training  to  a  non-aviation  crew- 
served  weapon  system  that  battle  rosters  crews  (for  example,  Ml 
tank) .  Crew  coordination  research  has  focused  primarily  on  the 
aviation  component  of  the  Army's  combined  arms  team.  Increasingly 
complex  ground  systems  exhibit  crew  error  patterns  similar  to 
aviation,  resulting  in  degraded  mission  performance  and  safety 
mishaps,  especially  during  night  operations.  It  would  be  useful  to 
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apply  the  aviation-based  crew  coordination  concepts  to  a 
combat  system  and  conduct  similar  research  to  assess  the 
in  mission  performance  and  safety. 


ground 

difference 
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Appendix  A 

Crew  Member  Self-rating  Questionnaire  (Extract) 


Crew  Coordination  Research 
Crew  Member  Exit  Questionnaire 


Introduction 

This  questionnaire  is  used  to  record  your  perception  as  a  crew  member  during  the  AH-64  Combat 
Mission  Simulator  sessions  conducted  during  the  previous  few  weeks  by  USAAVNC  and  USARL  You  will 
be  asked  to  think  about  each  of  the  important  dimensions  of  crew  coordination  addressed  in  previous  training. 
For  each  dimension,  you  will  be  asked  to  give  us  your  subjective  ratings  of  effectiveness,  difficulty,  and  style 
used  during  your  "battle-rostered"  versus  "mixed  crew"  missions.  These  ratings  are  described  below: 

Effectiveness  This  rating  reflects  your  own  view  as  to  how  well  you  performed  this  aspect  of  crew 

coordination  as  a  crew.  The  rating  is  made  on  a  7-point  scale,  ranging  from  a  low  of  1  (Very 
Poor)  to  a  high  of  7  (Superior). 

Difficulty  This  rating  reflects  your  own  view  as  to  the  level  of  difficulty  you  encountered  in  performing 
this  aspect  of  crew  coordination.  Again,  the  rating  is  made  on  a  7-point  scale,  ranging  from 
a  low  of  1  (Very  Easy)  to  a  high  of  7  (Very  Difficult). 

Style  This  rating  reflects  your  own  view  as  to  the  manner  in  which  you  performed  this  aspect  of 

crew  coordination.  In  this  case,  style  refers  to  how  explicitly  were  the  crew  coordination 
actions  carried  out  during  the  missions.  The  rating  is  made  on  a  7-point  scale  ranging  from  a 
value  of  1  (Consistently  Explicit)  to  the  opposite  value  of  7  (Consistently  Implicit).  Explicit 
and  implicit  styles  are  further  described  as  follows: 

Explicit  This  aspect  of  crew  coordination  was  conducted  in  an  open  and  observable 

manner,  using  positive  statements  and  acknowledgements.  Nothing  was 
taken  for  granted  regarding  the  other  crew  member's  understanding  of 
required  actions  or  statements.  Crew  coordination  was  accomplished  "by 
the  book"  in  accordance  with  previous  training. 

Implicit  This  aspect  of  crew  coordination  was  conducted  in  a  nonobservable  and 

subconscious  manner,  without  the  use  of  positive  statements  and 
acknowledgements.  The  other  crew  member's  understanding  of  required 
actions  or  statements  was  assumed  to  be  consistent  with  yours.  Crew 
coordination  was  accomplished  subconsciously,  based  upon  a  sense  of 
mutual  anticipation  that  develops  through  previous  flight  experiences. 
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RATING  FORM 


Cross-indexing  code:  _ (Last  4  digits  of  SSN) 

1.  Establish  and  maintain  flight  team  leadership  and  crew  climate 
Effectiveness  in  Performing  This  Aspect  of  Crew  Coordination 

Battle  Rostered  □  Very  Poor  □  Poor  □  Marginal  □  Acceptable  □  Good  □  Very  Good  □  Superior 

Mixed  Crew  □  VeTy  Poor  □  Poor  □  Marginal  □  Acceptable  □  Good  □  Very  Good  □  Superior 

Difficulty  in  Performing  This  Aspect  of  Crew  Coordination 

Battle  Rostered  □  Very  Easy  C  Easy  □  Somewhat  Easy  □  Neutral  O  Somewhat  Difficult  □  Difficult  □  Very  Difficult 

Mixed  Crew  □  Very  Easy  □  Easy  □  Somewhat  Easy  □  Neutral  □  Somewhat  Difficult  a  Difficult  □  Very  Difficult 

Style  of  Performing  This  Aspect  of  Crew  Coordination 

Battle  Rostered  □  Consistently  □  Mostly  □  Somewhat  □  Mixed  □  Somewhat  □  Mostly  □  Consistently 

Explicit  Explicit  Explicit  Implicit  Implicit  Implicit 

Mixed  Crew  □  Consistently  □  Mostly  □  Somewhat  □  Mixed  □  Somewhat  □  Moaly  □  Consistently 

Explicit  Explicit  Explicit  Explicit  Implicit  Implicit 


2.  Premission  planning  and  rehearsal  accomplished 
Effectiveness  in  Performing  This  Aspect  of  Crew  Coordination 

Battle  Rostered  □  Very  Poor  □  Poor  □  Marginal  □  Acceptable  □  Good  □  Very  Good  □  Superior 

Mixed  Crew  □  Very  Poor  □  Poor  □  Marginal  □  Acceptable  □  Good  □  Very  Good  □  Superior 

Difficulty  in  Performing  This  Aspect  of  Crew  Coordination 

Battle  Rostered  □  Very  Easy  □  Easy  □  Somewhat  Easy  □  Neutral  □  Somewhat  Difficult  □  Difficult  □  Very  Difficult 

Mixed  Crew  □  Very  Easy  □  Easy  □  Somewhat  Easy  □  Neutral  □  Somewhat  Difficult  □  Difficult  O  Very  Difficult 

Style  of  Performing  This  Aspect  of  Crew  Coordination 

Battle  Rostered  □  Consistently  □  Mostly  □  Somewhat  □  Mixed  □  Somewhat  □  Mostly  □  Consistently 

Explicit  Explicit  Explicit  Implicit  Implicit  Implicit 

Mixed  Crew  □  Consistently  □  Mostly  □  Somewhat  □  Mixed  □  Somewhat  □  Mostly  □  Consistently 

Explicit  Explicit  Explicit  Implicit  Implicit  Implicit 
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Appendix  B 
Exit  Interviews 


Open-ended  Questionnaire  and  Response  Summary  (Crew  Member) 

This  appendix  contains  both  the  Crew  Member  and  the  IP  Evaluator 
Open-ended  Questionnaire  and  Response  Summary. 


I .  Introduction 

This  form  is  to  be  used  to  structure  the  exit  interview  for 
the  participants  in  the  crew  coordination  research  project, 
October  -  November  1993.  Questions,  both  closed-ended  and  open- 
ended,  are  an  integral  part  of  the  project  data  collection  plan. 
Closed-ended  questions  are  issued  separately.  The  open-ended 
questions  attached  to  this  form  are  intended  to  be  conversation 
starters  and  are  not  meant  to  limit  free  discussion. 


Important 

There  are  no  "right"  or  "wrong"  answers.  We  are  asking  for 
your  honest  opinions  so  that  we  can  identify  the  relative 
contribution  of  battle  rostering  and  standardized  crew 
coordination  training  to  flight  safety  and  mission 
performance . 


II.  General 

1.  What  is  your  AH-64  crew  duty  position?  Pilot  CPG 

(Circle  one) 

2.  How  much  experience  in  the  AH-64  do  you  have  in  this  duty 

position?  _  hours 

3.  Approximately  how  long  have  you  been  battle  rostered  with 
the  crew  member  you  flew  with  during  the  research  project? 

Less  than  30  days  _  60  -  90  days  _ 

30  -  60  days  _  More  than  90  days  _ 

4.  Approximately  how  many  hours  have  you  flown  in  the  last: 

90  days?  _  hours  180  days?  _  hours 

5.  Approximately  what  percent  of  the  time  do  you  fly  with  your 

battle-rostered  crew  member?  _  percent 
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6. 


Cross-indexing  code:  _  (Last  4  digits  of  SSN) 

(Note:  Exit  interview  results  will  be  correlated  with  other 

measures) 

7.  Do  you  believe  that  you  performed  better  as  a  crew  with  your 
battle-rostered  crew  member  or  other  crew  member? 

Summary:  Mixed  opinion.  Crew  members  believe  that  factors  like 

individual  skills,  experience  level,  and  familiarity  with  other 
crew  members  contribute  more  to  crew  performance  than  whether 
crews  are  battle  rostered  or  mixed.  Aviators  stated  that  mixed 
crews  were  better  at  crew  coordination;  that  is,  more  cross 
monitoring  and  less  complacency. 

Specific  Comments: 

Definitely  better  as  a  battle-rostered  crew.  Another  crew 
member  disagreed. 

After  you've  flown  with  someone  for  awhile,  you  start 
thinking  in  synch  and  you  don't  have  to  tell  each  other  what 
to  do. 

There  is  better  flow  with  a  battle-rostered  crew  member;  not 
as  much  talking  in  the  cockpit.  Some  non-verbal  information 
[symbology]  is  shared  within  the  aircraft  system. 

The  first  few  flights  with  your  battle-rostered  crew  member, 
you  cross  monitor  more.  Initially,  you  try  to  cross  monitor 
to  sense  out  the  other  guy. 

With  your  battle-rostered  crew  member,  you  tend  to  be  a  lot 
more  implicit  in  your  coordination.  With  a  mixed  crew,  you 
have  to  discuss  more  things. 

-  Some  complacency  is  found  with  battle  rostering.  Flying 
with  other  crews  guards  against  complacency.  You  learn 
things  from  different  aviators. 

Coordination  was  better  in  the  mixed  crew,  even  though  we 
performed  adequately  in  both  cases.  The  other  crew  member 
in  a  mixed  crew  listened  better  than  the  battle-rostered 
crew  member. 

In  another  crew,  they  did  as  good  with  a  mixed  crew  as  they 
did  with  a  battle-rostered  crew. 

Overall,  there  was  no  difference  in  tactical  performance. 

-  Performance  depends  on  the  level  of  experience  of  the  crew, 
not  the  fact  that  they  are  battle  rostered.  Others  agreed. 
If  you  have  a  good  front  seater,  half  your  work  is  done 
regardless  of  whether  or  not  he  is  your  battle-rostered 
buddy.  There  is  more  frustration  with  a  weaker  front 
seater.  The  same  is  also  true  with  the  back  seater. 
Familiarity  is  a  big  factor.  If  skill  is  equal,  then 
familiarity  can  enhance  performance. 
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Tactical  performance  was  based  on  level  of  experience  and 
familiarity  with  the  equipment.  I  can  jump  in  with  anybody 
in  the  unit  and  fly  the  same  level  of  performance. 

It  depends  on  personalities  and  whether  or  not  you  have 
conflicts  in  the  cockpit. 

Battle  rostering  is  done  for  convenience,  not  because  we 
complement  each  other  in  personality  or  skill  level.  If  you 
complement  one  another,  you  fly  better;  but  this  is  not 
always  the  case  in  battle-rostered  crews. 

Conclusions : 

Verified  that  all  crews  were  more  verbal  since  they  were 
being  evaluated  on  crew  coordination. 

Confirmed  that  aircrews  are  concerned  about  the  potential 
for  complacency  within  battle  rostered  crews. 

Included  individual  skills,  experience,  and  familiarity  as 
dimensions  in  analysis  of  research  data. 

8.  What,  if  anything  did  you  do  or  notice  when  flying  with  your 
battle-rostered  crew  member  that  you  didn't  do  or  notice 
when  flying  with  another  unit  aviator? 

Summary:  General  agreement  that  there  is  more  explicit  crew 

coordination  and  deliberate  adherence  to  procedures  when  flying 
with  other  than  a  battle-rostered  crew  member. 

Specific  Comments: 

More  assertive  with  battle-rostered  crew  member. 

With  a  battle-rostered  crew  member  you  can  sense  the  urgency 
in  their  voice  better. 

With  a  battle-rostered  crew  member,  a  lot  of  the  procedures 
become  subconscious  (outside  of  awareness) ,  as  compared  with 
a  mixed  crew  (where  you  consciously  think  about  procedures) . 
The  same  thing  happens  when  you  don't  fly  for  ten  days.  The 
first  time  back  you  follow  all  of  the  procedures  more 
consciously. 

When  you  have  to  swap  out  with  another  crew,  you'll  not  be 
as  proficient  in  procedures.  You  perform  better  as  a 
battle-rostered  crew,  but  worse  when  you  fly  mixed  because 
of  the  lack  of  conscious  experience  with  following  the 
procedures. 

There  was  more  cross-monitoring  and  prompting  (providing 
information)  with  the  mixed  crew. 

They  didn't  rush  the  flow  of  the  mission  with  a  mixed  crew. 
More  deliberate  with  mixed  crew.  Didn't  "shoot  from  the 
hip"  as  with  battle  rostered  crew  member. 

More  information  is  passed  with  a  mixed  crew. 

You  tend  to  be  more  explicit  when  you  fly  with  a  mixed  crew. 
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Didn't  find  much  difference  when  flying  with  people  I  knew, 
as  compared  to  unfamiliar  crew  members  (lower  comfort  level 
with  strange  crew  member) . 

You're  more  apt  to  watch  someone  from  another  company,  as 
compared  with  someone  from  the  same  company  (even  though  I 
haven't  flown  with  the  other  guy  from  my  company). 

Different  companies  have  different  SOPs,  personalities,  etc. 
Some  found  it  easy  to  fly  with  another  company,  as  long  as 
standardized  terms,  etc  were  used. 

More  things  get  forgotten  or  blown  off  when  you're  flying 
the  aircraft  instead  of  the  CMS. 

Conclusion: 

Included  individual  skills,  experience,  and  familiarity  with 
other  crew  member  as  dimensions  in  analysis  of  research 
data . 

9.  In  whom  did  you  have  more  confidence,  your  battle-rostered 
or  nonbattle -rostered  crew  member?  Why? 

Summary:  General  agreement  that  confidence  in  other  crew  member 

is  more  a  function  of  skills,  experience  level,  and  familiarity 
than  whether  the  crew  is  battle  rostered  or  mixed. 

Specific  Comments: 

I  had  more  confidence  in  my  battle-rostered  crew  member 
(slightly  higher) .  This  dissipated  with  time  in  the 
cockpit . 

You  have  about  the  same  confidence  in  battle-rostered  and 
mixed  crew  members.  It  is  more  a  function  of  level  of 
experience  and  proficiency  of  the  other  crew  member  (it 
takes  you  at  least  one  mission  to  assess  this) . 

Others  stated  their  confidence  was  a  function  of  the  other 
crew  member's  rank  and  experience. 

Confidence  is  influenced  more  by  ability,  rather  than  battle 
rostering  versus  mixed. 

With  equal  skills,  your  confidence  is  about  equal.  You 
watch  weaker  pilots  more  closely  than  stronger  pilots. 

You  gain  confidence  in  others  when  they  fly  in  the  same 
flight  (same  company  but  not  the  same  aircraft).  You  don't 
get  this  when  you  mix  crews  across  companies.  You  tend  to 
know  who  is  who  in  the  company  level  unit. 

Your  confidence  may  be  affected  by  how  the  other  crew  member 
communicates  with  you  (do  they  reflect  -competence) . 
Confidence  can  lead  to  over  confidence  and  complacency. 

This  wasn't  seen  in  the  experiment,  but  is  seen  in  the 
field.  The  one  time  your  battle-rostered  buddy  doesn't 
perform  as  expected  comes  as  a  big  surprise. 

Conclusions: 
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Confirmed  that  aircrews  are  concerned  about  the  potential 
for  complacency  within  battle-rostered  crews. 

Included  individual  skills,  experience,  and  familiarity  as 
dimensions  in  analysis  of  research  data. 

Adverse  affects  of  complacency,  while  relatively  rare,  can 
be  very  costly. 

10.  Did  the  rank,  unit  position,  or  experience  of  the  other 
member  of  your  crew  affect  how  you  operated  as  a  crew  member? 

Summary:  Strong  agreement  that  the  experience  level  of  the  other 

crew  member  had  the  biggest  effect  on  how  individuals  operated  as 
a  crew  member. 

Specific  Comments: 

Rank  is  a  factor,  but  this  depends  on  who  you're  flying 
with.  Some  pilots  will  hold  back  criticizing  a  commissioned 
officer . 

-  Rank  can  be  a  factor  with  more  senior  officers,  but  not 
junior  officers. 

It  would  relieve  cockpit  stress  if  you  didn't  have  to  say 
"sir."  You  have  to  be  on  a  more  relaxed  basis  in  the 
cockpit. 

Rank  is  more  of  a  factor  outside  of  the  cockpit  (e.g., 
planning),  but  this  depends  on  individual  personalities. 

You  tend  to  be  real  formal  when  flying  with  your  rater. 

Unit  position  can  also  influence  responsibilities:  you  have 
to  cover  for  the  officer  by  doing  more  of  the  preflighting. 
Another  aviator  said  he  relied  on  his  battle-rostered  crew 
member  because  he  was  normally  the  mission  commander. 
Experience  is  the  big  factor,  not  rank  or  unit  position. 

I  let  the  other  crew  member's  experience  level  interfere 
with  my  PC  actions  in  decision  making  and  mission  flow.  I 
was  less  assertive. 

Experience  has  a  big  influence  on  your  confidence  level  and 
how  you  operate  as  a  crew  member.  When  you  fly  with  someone 
that  you  don't  know  how  experienced  they  are,  you're  just 
not  as  confident. 

The  other  crew  member's  level  of  proficiency  affects  how 
much  cross-monitoring  you  do. 

There  was  more  cross-monitoring  and  cross-checking  with  low 
levels  of  intracrew  confidence  (even  regarding  aircraft 
control) . 

Conclusion: 

Analyzed  research  data  to  identify  effect  of  experience 
level  on  crew  performance. 

Rank  intimidation  in  the  cockpit  is  a  persistent  problem. 
Crew  Coordination  Training  should  emphasize  the  need  to 
overcome  entrenched  cultural  courtesy  when  operating  as  an 
aircrew. 
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11.  Were  you  more  formal  or  informal  depending  on  whether  you 
were  flying  with  your  battle-rostered  or  nonbattle-rostered  crew 
member? 

Summary:  General  agreement  that  there  are  differences  with 

battle-rostered  crews  being  more  informal  and  mixed  crews  more 
formal  ("by  the  book").  This  difference  is  more  pronounced 
during  premission  planning  and  rehearsal  activities. 

Specific  Comments: 

More  informal  and  implicit  communications  with  battle- 
rostered  crew  member.  More  formal  and  explicit  with  mixed 
crew  member. 

Battle-rostered  crews  accomplish  mission  planning  and 
rehearsal  more  informally. 

A  mixed  crew  might  be  more  explicit,  but  not  more  formal. 
Mission  briefing  was  more  formal  for  both  battle-rostered 
and  mixed  crews . 

Some  thought  there  was  no  difference  in  formality,  whereas 
others  said  a  mixed  crew  caused  them  to  be  more  formal 
(explicit)  in  their  coordination  techniques. 

Mixed  crews  resulted  in  more  detailed  and  explicit 
exchanges . 

Attribute  more  formality  to  IP  evaluation  atmosphere  not 
battle-rostered  or  mixed  crew. 

There  was  a  tendency  in  the  experiment  to  be  more  formal, 
but  in  the  field  the  crew  briefing  would  be  much  shorter  and 
more  informal. 

Conclusion: 

Included  crew  member  self-rating  exit  interview  items  on 
crew  coordination  style  (explicit  versus  implicit)  in  the 
analysis  of  crew  performance  data. 

12 .  Were  you  able  to  identify  any  error  chains  during  your 
missions?  Were  you  better  able  to  identify  error  chains  with 
your  battle-  or  nonbattle -rostered  crew  member? 

Summary:  Mixed  opinion  regarding  ability  to  identify  and  break 

error  chains. 

Specific  Comments: 

No  differences  noted  by  most  of  the  aviators. 

You  don't  notice  the  error  chain  until  the  big  disaster 
happens . 

With  a  battle-rostered  crew,  you  can  see  things  building 
more  easily  and  intervene  quicker. 

Battle-rostered  crews  were  quicker  to  identify  error  chains 
because  you  had  a  better  basis  of  comparison. 
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More  flags  or  vocal  cues  are  recognizable  with  a  battle- 
rostered  crew. 

You  know  your  battle-rostered  crew  member's  strengths  and 
weaknesses  and  can  anticipate  errors.  With  a  mixed  crew, 
you  aren't  as  aware  of  where  errors  might  occur. 

There  is  more  willingness  to  challenge  one  another  with  a 
battle-rostered  crew,  as  compared  with  allowing  situations 
to  develop  with  a  mixed  crew. 

With  a  mixed  crew,  you  had  to  have  several  things  go  wrong 
before  you  began  to  see  a  pattern  and  intervene. 

With  a  mixed  crew,  you  tend  to  continue  the  mission  longer 
without  raising  an  issue.  Another  crew  member  stated  that 
he  would  be  quicker  to  ask  questions  with  a  mixed  crew. 

A  mixed  crew  is  also  less  willing  to  call  out  errors. 

With  a  mixed  crew,  I  had  more  of  a  tendency  to  make  a 
decision  when  things  were  going  wrong  (more  assertiveness), 
whereas  on  a  battle-rostered  mission,  I  held  back  and  waited 
for  a  discussion  to  ensue  before  a  decision  was  made.  This 
could  be  related  to  a  lack  of  confidence  in  the  other 
(mixed)  crew  member. 

I  was  more  assertive  in  making  a  decision  with  a  mixed  crew. 
Breaking  error  chains  depends  on  crew  member  assertiveness. 
The  error  chains  in  the  CMS  were  short,  as  compared  to  being 
in  a  more  dynamic  field  environment. 

Conclusion: 

Included  error  chain  observations  made  by  data  collection 
team  members  in  the  analysis  of  crew  performance  data. 

13.  What  is  (are)  the  advantage (s)  of  flying  with  your  battle- 
rostered  crew  member  now  that  all  aviators  in  your  unit  have 
received  the  Army's  standardized  crew  coordination  training? 

Summary:  Mixed  opinion.  Advantages  of  crew  familiarization 

achieved  by  battle  rostering  are  outweighed  by  standardization  of 
crew  coordination  training. 

Specific  Comments: 

There  are  advantages  of  familiarity  and  ability  to 
anticipate  and  cut  off  error  chains  (knowing  strengths  and 
weaknesses  of  your  battle-rostered  crew  member) . 
Battle-rostered  crews  spend  time  together  outside  of  the 
cockpit  and  develop  close  relationships.  You  know  what's 
going  on  in  their  life,  their  personality,  their  mood  that 
day;  all  beyond  just  flying  together.  You  can  sense  when 
your  crew  member  is  not  fully  functional  and  should  not  be 
flying.  When  stuff  starts  going  wrong,  it's  easier  to  talk. 
Battle  rostering  adds  more  familiarity  and  informality.  It 
takes  about  3-4  missions  to  gain  the  needed  familiarity. 
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Battle  rostering  might  provide  an  advantage  in  that  initial 
combat  mission.  Afterwards,  there  is  no  advantage  to  battle 
rostering. 

Battle-rostered  crews  are  more  informal  and  brief.  Mixed 
crews  operating  by  the  book  achieve  higher  levels  of 
performance  quicker. 

There  are  negligible  advantages  of  battle  rostering  once  you 
have  had  crew  coordination  training.  Standardization 
improves  performance  across  the  whole  unit. 

Crew  coordination  training  provides  a  baseline  of 
standardization. 

The  crew  coordination  "shared  experience"  of  the  simulator 
training  gives  you  better  mental  preparation;  even  for 
flights  in  the  aircraft. 

Mandated  battle  rostering  is  a  disadvantage. 

Conclusion: 

Incorporated  crew  member  opinions  into  battle  rostering 
policy  and  crew  coordination  training  policy 
recommendations . 

14.  What  is  your  overall  impression  of  the  Army's  practice  of 
battle  rostering  aircrews?  Do  you  have  any  suggested 
improvements  ? 

Summary:  General  agreement  that  formally  mandated  battle 

rostering  in  conjunction  with  crew  readiness  level  progression 
hampers  unit  flexibility  to  meet  mission  requirements.  Battle 
rostering  on  an  informal  level  is  preferred  over  a  formal  system. 
Specific  Comments: 

You  learn  from  other  aviators,  but  you  are  more  comfortable 
with  and  prefer  to  fly  with  your  battle-rostered  crew  member 
on  important  missions. 

Just  knowing  someone  in  your  company  gives  you  confidence, 
as  compared  to  flying  with  someone  from  another  company. 
Everyone  should  be  battle  rostered,  but  everyone  should  go 
through  the  crew  coordination  training.  The  training 
provides  effective  ways  of  operating  in  the  cockpit. 

Battle  rostering  builds  complacency  that  can  bite  you  some 
day. 

Battle  rostering  is  not  needed  because  crew  coordination 
training  forces  you  to  standardize. 

Crew  coordination  training  is  more  valuable  than  battle 
rostering. 

Battle  rostering  also  takes  away  from  the  unit's  versatility 
of  flying  with  mixed  crews.  Mixed  crew  flights  keep  you 
alert. 

It  is  not  worth  the  effort.  It  is  too  hard  to  rate.  In  the 
field,  you  won't  remain  with  your  battle-rostered  crew 
anyway.  It  should  be  done  informally,  but  not  required  in 
writing. 
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You  have  to  consider  unit  cohesiveness:  you  can  launch 
quicker  if  you  have  a  cohesive  unit. 

Battle  rostering  on  an  informal  level  is  preferred  over  a 
formal  system. 

There  are  already  too  many  other  constraints  on  matching 
crews  to  missions. 

The  formal  program  is  difficult  to  implement  in  the  unit. 
Units  are  hamstrung  and  discouraged  from  experiencing  the 
training  value  of  flying  with  different  aviators.  You 
should  be  able  to  fly  with  anyone  in  the  unit. 


Conclusion: 

Incorporated  crew  member  opinions  into  battle  rostering 
policy  and  crew  coordination  training  policy 
recommendations . 

15.  What  is  your  overall  impression  of  the  Army's  practice  of 
crew  readiness  level  (CRL)  progression?  Do  you  have  any 
suggested  improvements? 

Summary:  Strong  agreement  that  crew  readiness  level  progression 

is  an  unnecessary  administrative  requirement. 

Specific  Comments: 

Prefer  battle  rostering,  but  do  not  want  it  to  be  a  formal 
written  policy  for  CRL  progression. 

Eliminate  CRL  progression,  but  continue  to  fly  with  your 
battle  rostered  crew  member. 

CRL  is  an  unnecessary  paper  drill. 

CRL  progression  is  a  waste  of  time. 

Why  should  I  have  to  artificially  verbalize  standard 
phraseology  during  a  CRL  progression  check  ride  when  I  don't 
do  it  during  routine  missions?  There  is  already  too  much 
communication  in  a  mission,  and  it  makes  for  a  confusing 
cockpit.  Others  disagreed  with  this  comment. 

Conclusion: 

Incorporated  crew  member  opinions  into  battle  rostering 
policy  and  crew  coordination  training  policy 
recommendations . 
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Open-ended  Questionnaire  and  Response  Summary 

(IP  Evaluator) 


I .  Introduction 

This  form  is  to  be  used  to  structure  the  exit  interview  for 
data  collection  team  members  in  the  crew  coordination  research 
project,  October  -  November  1993.  Questions,  both  closed-ended 
and  open-ended,  are  an  integral  part  of  the  project  data 
collection  plan.  Closed-ended  questions  are  issued  separately. 
The  open-ended  questions  attached  to  this  form  are  intended  to  be 
conversation  starters  and  are  not  meant  to  limit  free  discussion. 


Important 

There  are  no  "right"  or  "wrong"  answers.  We  are  asking  for 
your  honest  opinions  so  that  we  can  identify  the  relative 
contribution  of  battle  rostering  and  standardized  crew 
coordination  training  to  flight  safety  and  mission 
performance . 


II.  General 

1.  What  was  your  job  on  the  data  collection  team?  (Circle  one) 

Evaluator  Data  Collector 
CMS  Instructor  Operator 
Data  Logger  Collector 
Workload  Data  Collector 

2.  Approximately  how  many  missions  did  you  observe  and/or 
collect  data? 

_  missions 

3.  Cross-indexing  code:  _  (Last  4  digits  of  SSN) 

(Note:  Exit  interview  results  will  be  correlated  with  other 

measures) 
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III.  Open-ended  Questions 

4 .  Do  you  believe  that  crews  performed  better  when  battle 
rostered  or  mixed?  Why? 

Summary:  Mixed  opinion.  Evaluators  believe  that  individual  and 

crew  proficiency  contribute  more  to  crew  performance  than  whether 
the  crews  are  battle  rostered  or  mixed. 

Specific  Comments: 

Battle-rostered  crews  had  well  established  crew  climates  and 
were  familiar  with  each  other. 

-  Mixed.  Mixed  crews  appeared  to  be  more  communicative  and 
less  complacent. 

-  Mixed  crews  definitely  coordinated  better,  but  it  is  hard  to 
say  that  this  resulted  in  better  mission  performance. 

-  Mixed  crews  talked  more,  but  overall  accomplishment  of 
tactical  objectives  was  more  a  function  of  crew  proficiency. 

-  No  significant  differences  were  observed  in  overall 
performance. 

-  There  is  a  fine  line  between  a  crew  coordination  problem  and 
an  individual  proficiency  problem  that  is  hard  to  tease  out 
in  evaluations. 

-  You  try  to  achieve  a  balance  in  proficiency  when  you  battle 
roster.  In  mixed  crews,  you  end  up  with  really  strong 
pairings  and  really  weak  pairings. 

Some  crew  members  incorrectly  perceived  that  lots  of 
communication  was  needed  to  be  evaluated  at  a  high  level. 

-  Verbal  communication  is  not  needed  when  the  AH-64  system's 
shared  symbology  provides  a  certain  level  of  nonverbal  cross 
monitoring. 

In  stressful  situations,  you  will  always  have  two  separate 
cockpits  and  very  little  communication. 

-  There  is  a  danger  that  standardized  call  outs,  required  in 
check  flights,  will  become  thoughtless  exercises  defeating 
their  intended  purpose. 

Conclusion: 

Identified  the  importance  of  analyzing  the  contribution  of 
crew  coordination  to  ATM  task  performance  as  a  means  to 
focus  on  proficiency  issues. 

Included  individual  skills,  experience,  and  familiarity  as 
dimensions  in  analysis  of  research  data. 

5.  What,  if  anything  did  you  notice  the  crews  do  when  flying 
with  their  battle-rostered  partner  that  they  did  not  do  when 
flying  with  another  unit  aviator? 

Summary:  General  agreement  that  battle  rostered  crews  tend  to 

communicate  implicitly  and  abbreviate  mission  planning  and 
rehearsal . 
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Specific  Comments: 

-  Implied  communication  and  implied  tasks  were  much  more 
prevalent  with  battle-rostered  crews,  as  compared  with  mixed 
crews . 

Battle-rostered  crews  used  much  shorter  statements  to 
communicate  ideas. 

-  There  was  a  greater  comfort  level  with  battle  rostering,  but 
this  did  not  produce  performance  differences. 

Battle-rostered  crews  knew  each  other  in  the  flight 
environment. 

Battle-rostered  crews  tended  to  abbreviate  mission  planning 
with  comments  such  as,  "Like  we  always  do."  When  pressed 
for  details,  for  example,  emergency  procedures  and  egress 
from  a  battle  position,  crew  members  did  not  have  a  common 
understanding. 

I  had  the  sense  that  battle-rostered  crew  members  spent  time 
contemplating  what  each  other  was  thinking  or  was 
responsible  for  before  taking  action. 

-  Mixed  crews  were  more  explicit  in  communications,  both 
internal  and  external,  and  making  task  assignments. 

-  Mixed  crews  conducted  more  detailed  planning  and  crew 
briefings . 

Conclusion: 

Included  individual  skills,  experience,  and  familiarity  as 
dimensions  in  analysis  of  research  data. 

-  Abbreviated  planning  and  omission  of  "by  the  book"  callouts 
are  persistent  problems  to  be  addressed  by  crew  coordination 
training . 

6.  Did  the  rank,  unit  position,  or  experience  differences 
between  crew  members  affect  how  crews  operated?  Did  it  affect 
crew  performance?  Examples? 

Summary:  General  agreement  that  rank  and  experience  affected 

crew  performance. 

Specific  Comments: 

-  Yes,  rank  and  experience  differences  were  important:  two 
classic  examples  were  seen  in  the  flights. 

-  Lack  of  assertiveness  on  the  part  of  the  junior  crew  member 
was  the  problem. 

-  PCs  didn't  want  to  tell  higher  ranked  or  more  experienced 
crew  members  about  errors . 

Yes,  excessive  professional  courtesy. 

-  In  combat,  though,  you  need  a  definite  cockpit  gradient  in 
order  to  make  unpopular  decisions. 

Did  not  notice  any  effect  at  all. 
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The  biggest  factors  that  influenced  the  outcome  of  the 
missions  were  a)  the  pilot's  PNVS  flight  proficiency,  b)  the 
CPG's  weapon  system  proficiency,  and  c)  both  crew  member's 
situational  awareness.  All  three  factors  correlate  directly 
with  experience. 

Conclusions: 

Analyzed  research  data  to  identify  effect  of  experience 
level  on  crew  performance. 

Rank  intimidation  in  the  cockpit  is  a  persistent  problem. 
Crew  coordination  training  should  emphasize  the  need  to 
overcome  entrenched  cultural  courtesy  when  operating  as  an 
aircrew. 

7 .  Were  crews  more  formal  or  informal  depending  on  whether  they 
were  battle  rostered  or  mixed?  Examples? 

Summary:  Strong  agreement  that  there  are  differences  with 

battle-rostered  crews  being  more  informal  and  mixed  crews  more 
formal  or  "by  the  book".  This  difference  is  more  pronounced 
during  premission  planning  and  rehearsal  activities. 

Specific  Comments: 

Battle-rostered  crews  were  more  informal,  complacent,  and 
overlooked  things.  Mixed  crews  were  more  formal,  attentive, 
and  did  not  overlook  things  as  much. 

In  battle  rostering,  when  nothing  gets  said  (because  it  is 
implicitly  understood),  nothing  ultimately  gets  done. 

Mixed  crews  were  definitely  more  formal  in  planning  and 
rehearsal  and  crew  briefing. 

Mixed  crews  were  more  formal  in  procedural  aspects. 

Mixed  crews  seemed  to  ask  more  questions  of  each  other 
during  the  flight. 

Conclusions : 

Included  self-rating  exit  interview  items  on  crew 
coordination  style  (explicit  versus  implicit)  in  the 
analysis  of  crew  performance  data. 

Crews  should  be  repeatedly  reminded  to  operate  "by  the  book" 
and  not  make  a  lot  of  assumptions  about  their  fellow  crew 
member's  responsibilities,  duties,  and  actions. 

8.  Did  you  observe  any  significant  difference  between  battle- 
rostered  and  mixed  crew  performance  of  specific  ATM  tasks?  Which 
ATM  tasks? 

Summary:  Mixed  opinion  regarding  significant  difference  in  crew 

performance  of  ATM  tasks  other  than  mission  planning. 
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Specific  Comments: 

Mission  planning  was  different.  Mixed  crews  were  definitely 
more  detailed  and  deliberate. 

Mixed  crews  performed  better  in  terrain  flight  navigation. 
Battle-rostered  crews  performed  firing  position  operations 
better  than  mixed  crews. 

-  Mixed  crews  appeared  to  perform  better  in  the  following  ATM 
task  areas:  a)  PNVS  flight,  b)  navigation,  c)  CPG  weapon 
system  operations,  and  d)  instrument  approach  procedures. 

Conclusion: 

Considered  comments  in  analysis  of  ATM  task  research  data. 

9.  Did  you  notice  any  difference  in  the  type,  frequency,  and/or 
duration  of  crew  error  chain  patterns  between  battle-rostered  and 
mixed  crews?  Examples? 

Summary:  Mixed  opinion  regarding  ability  to  identify  and  break 
error  chains. 

Specific  Comments: 

-  Quicker  detection  of  error  chains  with  battle-rostered  crews 
was  purely  a  perception  on  the  part  of  the  crews.  In 
reality,  battle-rostered  crews  did  not  catch  errors  any 
sooner  than  mixed  crews. 

No  difference  in  time  to  resolve  errors. 

When  mixed  crews  made  an  error,  it  seemed  to  propagate  into 
larger  or  additional  errors. 

Conclusion: 

Included  error  chain  observations  made  by  data  collection 
team  members  in  the  analysis  of  crew  performance  data. 

10.  What  is  (are)  the  advantage (s)  of  battle  rostering  aircrews 
after  all  aviators  in  a  unit  have  received  the  Army's 
standardized  crew  coordination  training? 

Summary:  General  agreement  that  battle  rostering  helps  reduce 

risk  by  balancing  differences  in  individual  experience  levels 
within  units. 

Specific  Comments: 

There  are  no  advantages  to  battle  rostering  except  to 
balance  experience  levels. 

Accommodates  large  dispersions  in  individual  proficiency 
within  a  unit. 

Familiarity  with  other  crew  member  personality  traits  and 
behaviors  in  the  aircraft  environment. 

Battle  rostering  will  help  a  deploying  unit  transition  into 
combat. 
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Conclusion: 

Incorporated  crew  member  opinions  into  battle  rostering 
policy  and  crew  coordination  training  policy 
recommendations . 

11.  What  is  your  overall  impression  of  the  Army's  practice  of 
battle  rostering  aircrews?  Do  you  have  any  suggested 
improvements? 

Summary:  General  agreement  that  formally  mandated  battle 

rostering  adversely  affects  unit  safety  and  flexibility  to  meet 
mission  requirements. 

Specific  Comments: 

Battle  rostering  is  good.  Keep  battle  rostering  along  with 
crew  coordination  training. 

Right  now,  battle  rostering  is  an  Army  policy  that  is 
detracting  from  unit  safety  rather  than  improving  it. 

Remove  all  mandatory  requirements  for  battle  rostering  from 
regulation.  Mandated  battle  rostering  is  a  disaster.  Don't 
mandate  it  under  a  title  or  it  will  be  interpreted  as 
regulation. 

Battle  rostering  doesn't  allow  the  unit  to  maximize 
proficiency  and  mission  performance.  The  unit  is  more 
important  than  the  individual  crews. 

Battle  rostering  is  little  more  than  a  means  to  save 
training  dollars.  Crew  members  should  be  proficient  in  both 
AH- 64  stations  and  should  be  given  sufficient  hours  to  do 
so.  Mixed  crews  offer  greater  flexibility  and  safety. 

Battle  rostering  impedes  unit  mission  performance  by 
limiting  crew  selection  and  mission  assignments  when  rigidly 
enforced. 

If  implemented  informally  as  part  of  risk  assessment  and 
unit  training  management,  give  it  another  name  like  "crew 
management  or  crew  selection  process." 

Drop  all  reference  to  the  term  battle  rostering  and  let  IPs 
and  commanders  make  crew  selections  based  on  crew  member 
experience,  proficiency,  and  personality. 

Battle  rostering  should  be  informal  and  left  up  to  the  unit 
commander  to  implement. 

Conclusion: 

Incorporated  crew  member  opinions  into  battle  rostering 
policy  and  crew  coordination  training  policy 
recommendations . 
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12.  What  is  your  overall  impression  of  the  Army's  practice  of 
crew  readiness  level  (CRL)  progression?  Do  you  have  any 
suggested  improvements? 

Summary:  Strong  agreement  that  crew  readiness  level  progression 

is  an  unnecessary  administrative  requirement. 

Specific  Comments: 

-  Requirement  is  a  waste  of  time  and  paper  and  should  be 
deleted. 

Replace  CRL  progression  with  crew  coordination  training. 

-  The  ripple  effect  of  replacing  one  crew  member  makes  the  CRL 
progression  impractical. 

Consider  replacing  CRL  progression  with  crew  coordination 
training,  gunnery  table  8,  and  the  crew  conduct  of  fire 
trainer  (C-COFT) . 

-  CRL  progression  is  not  needed  once  crew  members  are  trained 
in  the  crew  coordination  basic  qualities. 

CRL  progression  is  creating  problems  and  will  continue  to 
cause  problems  in  the  future. 

Crew  coordination  training  negates  the  need  for  CRL 
progression . 

Conclusion: 

-  Incorporated  crew  member  opinions  into  battle  rostering 
policy  and  crew  coordination  training  policy 
recommendations . 
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